Including the Community Shield, the Blues have conceded 11 goals in six games so far this season, culminating in the three at the Bernabeu on Tuesday that cost them a famous Champions League away win over Real Madrid.
"For two years we were the best defence in the Premier League," he said.
"This year we have been average.
"Probably the big problem was we didn't get to work properly in pre-season. Since then we lost some players and have had an international break.
"But if we are going to improve, we need to work."