Palace striker Glenn Murray got his toe to the ball before Vorm and his run through on goal was only ended by Chico Flores' desperate challenge.
Although contact was possibly just outside the box it was a red card for Chico and Murray converted from the spot to hand Palace reward for their second-half dominance.
"I hoped it would bounce further than it did," said Vorm. "I had to do better than what I did, but Chico had to make the challenge.
"I thought the foul was outside the box but it's a red card, a penalty and a goal, so we had to fight for a point just in the end."
For breaking stories and all the great banter like us on Facebook: facebook.com/tribalfootball