The Belgian was poor in Monday's loss to Manchester United, missing a huge chance to break the deadlock in the first-half.
Asked after the game if Batshuayi lacked confidence, Lampard said: "Confidence is part of football.
"The only way you get over confidence [issues] is by dedication and hard work.
"Every player will go through it no matter who you are. The only way to get through it is hard work."