The Eagles were typical of a team in a relegation battle when Mama Diame fired home after being left unmarked at a corner.
10 minutes in the second half, Palace had their chance at an equaliser.
Towering striker Christian Benteke was held back by Magpies defender Ciaran Clark when a cross was delivered into the box.
Milivojevic then stepped up to take the penalty, which despite Karl Darlow's best efforts, ended up in the back of the net.
Palace nearly had the winner as well in the 88th minute but Clark cleared a Benteke header off the line.
The result leaves Palace in 14th place, and three points off the relegation zone. Meanwhile, Newcastle are just one point off the relegation zone in 16th.