He knows his team did not start well against champions City last Sunday and started to show some fight only in the second half - when they were already 2-0 down.
Mertesacker, whose team beat Swansea in the FA Cup on Wednesday, said: "If you want to be in the top four you have to compete with Chelsea, Man City, Man United and Tottenham.
"These teams are in this zone. We need to win against those four as much as we can - it is a good point to show how we are.
"We need our character on Sunday. That is the most important thing. In the second half against Manchester City, we showed our character, which was much better than the first half.
"We were a bit scared against Man City from the beginning. We were not that confident.
"The second half was much better. With a bit of luck we could have scored in two or three situations.
"But we have to take results - the FA Cup win against Swansea was vital to prove that we were ready to compete."