Rosenior signed a five-and-a-half-year deal after arriving from French club Strasbourg but managed just 11 wins in 23 matches across all competitions.
The 3-0 loss to Brighton was the final straw for the Englishman as he became the latest coach to be ruthlessly sacked by the Chelsea board.
Speaking to Sky Sports, Neville revealed that he was stunned to see Rosenior being sacked so soon especially after reports claimed he had been backed by the board for up to a year.
"I'm not shocked Liam's leaving the club. I'm shocked he's leaving today. I thought it would probably happen at the end of the season. I thought they would persevere until the end of the season after giving him a six-year deal. It's not a reflection on Liam at all. He's lost a lot of football matches in the last few weeks, and he'll be disappointed with that.
"But it's time for the owners, sporting directors and players to reflect enormously on their role in what's happened in the last few weeks. Their fans will be absolutely disgusted by the way this has gone. The owners have got it badly wrong.
“I said at the time, they've got a group of talented players, young sporting directors in respect of the experience relating to Europe, you've got an ownership that's young in respect that it's only two or three years in. You need some experience somewhere in the club."
Rosenior leaves the club seventh in the Premier League, seven points off fifth-placed Liverpool, who have a game in hand in the race for Champions League football which Chelsea now look to be far behind.