Cole says Lampard needs Willian back to match fitness and ready to help them bounce back from just one point in their first two Premier League games.
Antonio Rudiger and Loftus-Cheek are also sidelined, and although they are facing a longer absence with injury, Cole sees them both helping Lampard's side further when they return.
Cole told Sky Sports: "Rudiger, Ruben [Loftus-Cheek] will add to this, we're talking about not having enough goalscorers… maybe he [Loftus-Cheek] can give something different.
"So, yes, they are missing players. Willian has obviously been away. He looked a little bit rusty today.
"Frank needs all these players sharp now. Yes, it's only two games in but he's already getting judged.
"So he's going to have to be motivating these guys in a different way and say: 'Hey, you better hurry up and start to get fit because I need you'."