The Red Devils looked set to claim all three points before left-back Paul Dummett scored a last-minute goal to earn a point for the Magpies in what was n entertaining affair.
United led 2-0 thanks to an early penalty from Wayne Rooney and a fine finish from Jesse Lingard before Georginio Wijnaldum volleyed home and Aleksandar Mitrovic converted from the spot after being wrestled to the ground by Chris Smalling.
Rooney supplied a stunning finish to see United back in front before Dummett's late deflected effort denied the visitors maximum points.
Van Gaal said: "We threw it away. You have to finish the game much earlier.
"The referee gave a penalty and I think it was a duel - I have seen the video and you can see Mitrovic was pushing the head of Smalling. When you both do that you cannot decide which is worse, I think, but the referee did not lose our game.
"We lost it by ourselves."
The Dutch manager was also critical of his players' failures to finish the game, adding: "They have to finish of course but it is always difficult.
"Rooney also had a very big chance in the first half. We could have scored six goals here, I think, there were big possibilities so when you then lose two points then that is very sad.
"Also, everybody knows it is our own fault. It was not the referee or the opponent. We did it ourselves."