Allardyce was delighted to see Mark Viduka open his Magpies account with his 150th goal in British football, and with Charles N'Zogbia's earlier stunning strike.
However, he was hugely disappointed with the two his side conceded as defensive frailty - something which will not be improved by injuries to Stephen Carr and Peter Ramage - once again cost them dear.
Allardyce said: "We have conceded two and that is totally out of order, especially when you have been in front twice.
"It is just unacceptable, absolutely unacceptable. If Newcastle United are to be successful, we have got to cut that out.
"We are bitterly disappointed that we have not clinched the three points.
"The lads are really, really disappointed in the dressing room and know they should have hung on to a lead they got twice.
"Once you get in front twice whether you are playing home or away, you should never surrender it.
"Perhaps the first time is an excuse, but not the second."
Opposite number Gareth Southgate was delighted with the application of his players, especially defenders Jonathan Woodgate and Luke Young, who started with only a few days of training under their belts after injury.
He said: "I am pleased with the way we played. For a local derby, we tried to pass the ball and we tried to attack them.
"We went a goal down against the run of play and we showed great character to come back twice.
"All the qualities that were on display from my team, I was very proud of them.
"We had a side that looks a good side, and when they are fully fit and firing, there were a lot of good signs from us.
"What pleased me more than anything was we had lads go out and play who probably had no right to be on the field.
"They committed themselves for the short and for their team-mates and as a manager, that makes you very proud."