Santi Cazorla's goal separated the two teams.
O'Neill was left scratching his head after seeing his team come up only just short.
He said: "It was a great game from start to finish. The tempo was very high early on and it remained so.
"Obviously Arsenal, a top-quality team, caused us a lot of problems in the first half but I also thought we looked dangerous as well.
"But in the second half, we just came roaring at them, and that was even before the man had been sent off.
"Seriously, looking back at a couple of the chances, I just don't know how we didn't score."