Faye was red-carded for a two-footed tackle in the 35th minute - and two minutes later Boro took the lead through a superb Afonso Alves free-kick.
"We were well in the game until the sending off," said Pulis.
"I haven't seen it on TV yet, so I won't say anything about the sending off.
"But it was a great free-kick that followed and you can't do anything about that."
The visitors levelled through a Justin Hoyte own goal, but Boro snatched the points with an 85th-minute winner.
"The players had to work so hard out there to get something out of the game," added Pulis, "and I thought we would come away with something after we scored and took the steam out of them.
"To play a long time with 10 men and get an equaliser, then get punished like that after a mis-hit shot drops to somebody in the box is tough on our lads.
"Good players will move the ball around against 10 men and they did that to us, but our lads kept on working.
"Of course it's hard to take because I want to win games and don't like losing, irrespective of how you play."