Olivier Giroud opened the scoring early at Selhurst Park after being set-up by Willian, who was given a free run into the box as Gary Cahill went down with a match-ending injury.
Christian Pulisic made it 2-0 when he jinked past Joel Ward on the edge of the box to find a yard of space before firing a rasping strike through the inside post of Vicente Guaita.
But a stunning 30-yard strike from Wilfried Zaha got Palace back in the game to set-up a fearsome second-half.
The Blues' recent defensive problems were highlighted again when Christian Benteke was allowed to peg one back only minute later.
Speaking after the game, Lampard said: "There were moments in the game where our game-management wasn't great.
"At 2-0 we can dominate that game and finish it. When we went 3-2 up we had chances to finish the game and were sloppy.
"But at this point, three points is the critical thing."