The Gunners won 2-1 at St James' Park thanks to Gabriel's 96th minute header.
Arteta said afterwards: "Unbelievable feeling. That's what football is about and a team. When you get what you deserve right at the end, it's not a better feeling than that
"I'd seen the action very clear and fast, so I thought (the referee) wasn't going to intervene but I don't know because obviously we had the example of what happens in the first half with a penalty.
"Regardless of that, I'm so proud of the team, the way we played, the way we competed, the way we understood the game, how we reacted to very difficult situations and decisions and still found a way to win the game with the players coming from the bench, exceptional.
"There are a few things that might be unpredictable with the line-up or the formation or whatever. There are certain things that are so predictable, what you have to do here to win a football match. We certainly did that from the start to the end and on top of that, we added today so much quality from various ways to generate the amount of chances that we did. At the end we found a way to win it when I think we fully deserve to win the game."
Next level
Arteta added: "We discussed to go to the next level, first of all you have to learn from the past and certainly we take some lessons and very different and sore moments in this ground.
"Today was an opportunity that the game brings you again in a really important week in the Premier League after all the tough places that we've already been very early in the season to show who we are, who do we want to be, our ambition and the way that we want to play. I think the team has done that today in a remarkable way."