Roy Hodgson's side held a two-goal lead shortly after half-time, and would have climbed all the way up to 13th place with a win.
And, with 91 minutes on the clock, Matic smashed in a dipping shot from 25 yards to consign Palace to another week in the drop zone and move United above Liverpool.
Matic said: "After the second goal it was very difficult for us to come back, we showed character and did exactly what the manager wanted us to do.
"He said at half-time we had to do more. I'm happy I've scored my first goal for Manchester United, I know it's not my job but I'm glad it got us the three points."
He continued: "We are improving, we need to do more for sure. In the smaller games we drop some very important points, and we can't lose those points like at Leicester away.
"In these kind of games if we can manage to win we can compete for the title. The quality is there but we have to be honest, we need to do more.
"There are five or six teams in this division that can compete for the title. We will have a lot of confidence, once you win a game the supporters have confidence in you and it makes it easier."