The Hammers scored first when Matt Jarvis bravely headed home in the 40th minute but four minutes later, they gave up a goal when Stewart Downing hooked the ball straight to Santi Cazorla from near their defensive corner.
The Spaniard immediately found Lukas Podolski for an equaliser before Thomas Vermaelen played a long ball into the area for Olivier Giroud, who controlled superbly, before firing between the legs of goalkeeper Adrian in the 55th minute.
Arsenal's comeback was complete just over ten minutes from time when Podolski scored his second and the end result was something that did not sit well with Allardyce who criticised his players for undoing all their earlier hard work.
"We shot ourselves in the foot," he said.
"Doing so well to get the first goal, we then could not get into half-time 1-0 up, which was a big disappointment.
"We were then looking comfortable and the second goal sickened me off because the performance meant nothing after that.
"All of our hard work has been wasted."
For breaking stories and all the great banter like us on Facebook: facebook.com/tribalfootball