Should Arsenal win their lunchtime kick-off at struggling Wigan on Saturday, then they would, temporarily at least, leapfrog over Tottenham and back up into the top four.
"If you play for a team like Arsenal, there are always going to be people outside you who are going to question the manager and the team," Vermaelen said.
"The thing is, you have to stay united and work hard on the training pitch because we are the only ones who know what's going on in the team. What people say outside is just part of the job."
The Gunners captain added: "In the league, it is not going that badly.
"I agree that what we did in Bradford was a big disappointment because we want to win those [cup] games as well, but in the Premier League, it is all really close for a Champions League spot and at the moment we are chasing the top four."