An injurye-time strike from Fabio Tavares denied Luton three points as their League One promotion hopes continue to falter.
Fans chanted “you are embarrassing” at the players and Wilshere was seen trading insults with one angry fan.
Arsenal hero Wilshere had to be dragged away by Luton staff and players so to avoid things escalating.
Afterwards, the Hatters manager said: “First of all, I understand their frustrations, I do, I’ve said that a lot, but right now we need them.
“It’s not for a lack of the players trying, they’re giving everything every single day. We work really, really hard to try and win games, to try and make supporters happy.
“I think you can see that the players are really lacking belief, they’re lacking confidence.
“It’s difficult then when in the first minute there’s booing, but right now, everyone needs to be together, everyone needs to stick together. We have no right just to turn up and win games and expect that we can just do that as we can’t, you can clearly see that at the moment that is not the case.”