The Gunners struck in injury-time to earn the three points.
"I feel sick, I'm not going to lie," Simpson told Newcastle's website. "Last week it was us who got the late goal (against Sunderland).
"We were on a high and it felt good. This week, after the effort we put in, it's sickening to come here, play our hearts out and concede like that. It hurts.
"I wish I could have got back but I got cramp. I feel more gutted for me than for the lads; they're not saying it's my fault but I couldn't get back. I just cramped up, I think with all the work we put in. It was in my position at the end."