The Potters let the league leaders in after just three minutes when Michael Carrick scored before conceding a penalty which was converted by Robin van Persie with about 25 minutes remaining.
Both goals were rued by Pulis who told Sky Sports: "The first goal kills us. It was a poor goal to concede. The lads have worked well this week and to concede a poor goal - that's the last two home games we've conceded really quickly, the first time the opposition has been in our box.
"The second goal again, the game was tight and that kills it. If Wilko (Andy Wilkinson) stays on his feet, he doesn't concede the penalty."