Jon Walters edged the Potters ahead in the 67th minute when he converted from the penalty spot after he had been felled by Cheick Tiote in the area before Yohan Cabaye levelled it up with a brilliant free-kick five minutes later.
Papiss Cisse bobbed up to produce a 92nd minute winner for the Toon to force a result which clearly did not sit well with Pulis who feels a few incidents could have gone the way of his Stoke side.
"We are desperately disappointed. We could have had a penalty in the first half; the incident where Cisse was involved with Ryan (Shawcross) and Willo (Marc Wilson) - on another day, something could have happened with him and he might not even have been on the pitch," he said.
"The referee deemed that it was alright. Fine, if he's still on the pitch, we should defend better.
"But I am disappointed, absolutely distraught with the result, but the performance and the effort and the commitment of the players was absolutely first-class."