Darryl Janmaat opened the scoring after just three minutes, and it looked as though the Hornets would walk away with a handy away with before Tom Cleverly was given his marching orders after 87 minutes.
Buoyed by the extra man advantage, the hosts surged forward and found their equaliser through Bakery Sako.
James McCarthur then scored in stoppage time to help Palace temporarily climb out of the relegation zone.
Speaking after the match, Eagles manager Roy Hodgson was delighted with the win.
He said: "We got a late winner against Stoke as well and had disappointment on Saturday, so we've gone through the full range of emotions.
"We've done well in the late stages of games, so I've got to commend the players' attitude and determination to keep going even when things don't look like they're going our way.
"The goals we're scoring are often well-worked and Wilfried Zaha is often the man behind them, so we're delighted with his contribution.
"It is a huge three points. Where we are in the table every three points is unbelievably worked for.
"The moment we think our problems are over is the moment our problems will begin. We have a mountain to climb and it's wonderful to win but we're still very close to the bottom of the table because we haven't quite made up from getting zero points from the first seven games.
"It is stressful but if I didn't think I was able to take that stress then I wouldn't have taken up the offer to come and work at this club. The daily work with the players is as enjoyable as anything I've done in my career."