Chelsea's fans "You're not welcome" towards Benitez after Graham put Swansea's victory beyond doubt in the dying minutes and the Blues faithful again chanted the name of Spaniard's predecessor Roberto Di Matteo throughout.
They barracked Benitez for failing to introduce Ba earlier after Fernando Torres put in perhaps his worst performance of a woeful season.
The fans also hit out at Benitez for his failure to start the in-form Lampard, but Benitez refused to be drawn on the home crowd's criticism.
"You analyse all these things because we lost," Benitez responded when asked about the criticism from the stands.
"We have to concentrate on our jobs."