The Gunners conceded in stoppage time in their 2-1 loss to Watford on Saturday.
And Mertesacker believes that the problems lies in lapses of concentration, rather than a complete lack of mental strength.
"We need to be mentally tougher in front of goal. We create the chances, but sometimes we need to switch on quicker," said Mertesacker.
"I believe we have that mental toughness but we need to switch on and do our thing. We were not good enough against Watford because we missed a chance and the majority of players were still thinking about that chance. Those things, switching on, need to be quicker.
"We had everything in place. We scored from a set-piece away from home, we protected the goal well for the first 60 minutes, we had a couple of chances to go further ahead - but we didn't.
"Then you need to be really resilient in those tight moments, and make clever decisions, but that's what we lacked basically. We want to be more clever in defending our own goal."