The Blues have conceded nine goals in five Premier League games so far this season.
Ahead of Tuesday's Champions League meeting with Sevilla, Azpilicueta says his teammates are aware they must work collectively to stop conceding goals.
"What is clear is that if we want to fight for everything we have to improve defensively," Azpilicueta told the pre-match press conference.
"This is a problem we have to fix collectively and individually. Sometimes you can get away with it but you cannot rely every game on having to score three goals.
"You have to dig in and get a clean sheet. It's very important for the confidence of the team to have this solidity. We have to start getting this consistency of not conceding goals. When you have the solidity defensively, it gives a boost to the whole team, to the attacking players to create chances.
"When you are defensively weak sometimes we feel like we have to score three or four goals every game. As a defender, I feel bad when you concede so many goals. We have to improve our personal mistakes and collectively take better decisions.
"Through the season you are not going to be playing beautiful football and winning three or four-nil every game. When you don't you have to get results."