Leading 3-2 from the first-leg, Arsenal won 1-0 at the Emirates thanks to Kai Havertz's injury-time winner.
Merson later said on Sky Sports: “I’m flabbergasted. I can’t believe what I’ve just watched.
“Chelsea aren’t a bottom-five team. They have World Cup winners.
“(Wesley) Fofana is crying. He should be crying because they never had a go. They’ve gone out with a whimper in a semi-final.
“It hasn’t worked. Go out in a blaze of glory, don’t go out like that.”
In response, Chelsea manager Liam Rosenior stated: said: “I’ve been a pundit. It’s easy. It’s easy in hindsight.
“If I go and attack the game, press really high and we can see two goals early, everyone says, ‘what’s he doing?’
“That’s the reality of my job. The reality of my job is if you lose games, you’ll be criticised.
“If you win, you’re a genius. It’s normally somewhere in between."