They are not convinced by his tactics or his ability to make the right decisions when things are going against them.
Villas-Boas, 34, was hired to mount a twin-pronged assault on the Champions League and Premier League.
It is the worst start the Blues have made to a season since Roman Abramovich took over in 2003. The word from the Chelsea hierarchy is Villas-Boas will be given more time, especially as owner Abramovich paid £13m to bring him in from Porto.
But Abramovich will not allow the decline carry on indefinitely.
A dressing room insider said: "It's at times like these that players look to the manager for strength and leadership but they don't see it coming from Villas-Boas.
"He is only 34 and perhaps the club are wondering if they made a mistake in bringing in such a young boss.
"There was a lot of respect for Ancelotti but there is not the same feeling for Villas-Boas at the moment."