Then Michael Turner rescued a share of the spoils for Chris Hughton's side with a late equaliser.
Lambert said: "I think the sending off changed a lot. That changed the game.
"At first thought I thought it was a sending off, because when you commit a foul and you go to do something, then you put yourself in a position where you can get the red.
"But, looking at it again, I think Elliott, too, has had hold of Joe. It was a bit of an even contest and I think the two of them were at it.
"I haven't had the chance to talk to the ref (Phil Dowd) about it yet. I've only just watched a tape of it.
"But I thought the team were brilliant with the way they defended strongly after that."