Australia international Mark Viduka rescued the Magpies with a double on his return to the starting line-up as the Rams made the most of yet more poor defending to twice take the lead at St James' Park.
Allardyce saw Viduka's team-mates fail to match his standards in front of goal with the visitors pinned back inside their own penalty area for long periods as Derby very nearly completed a double over the only team they have beaten in the Barclays Premier League this season.
The 53-year-old said: "I am disappointed, disappointed that we had to come back from a goal down and disappointed that we had to come back from 2-1 down when we clearly had enough opportunities to have buried any game.
"But we encouraged the opposition to have something to hang on to and then go right to the very end.
"Even though we nearly won it 3-2 at the death, we still have two clear problems to resolve: one is our basic defending today allowed the opposition to have two shots on target and score two; and we had numerous chances to convert more than the two we scored.
"Mark Viduka showed up the way forward with his finishing because he had three opportunities and converted two, and the first of them was our most difficult chance of the day.
"But the others went begging and ended up giving the opposition something to hang on to on two occasions."
Derby got off to the best possible start when midfielder Giles Barnes silenced the home crowd with just six minutes gone with a drilled low shot which beat Shay Given and nestled in the bottom corner.
It was just his side's second league goal on the road this season but although they defended it with real determination, it never looked like being enough.
However, the visitors survived until the break and then within seven minutes of the restart, got their noses back in front when Kenny Miller, whose goal clinched their home victory over Newcastle in September, finished off a route one attack with aplomb.
Newcastle laboured in the search for a second equaliser and had to wait until the 87th minute for Viduka to pounce on Darren Moore's weak header to level again.
Substitute Emre might have snatched victory deep into injury time, but saw his effort fly just wide, although that would have been tough on the spirited Rams.
Derby boss Paul Jewell said: "When you come so close to the end of the game and you concede a late goal, especially in the manner it was conceded - if they had scored a great goal, fair enough, but the two goals Newcastle scored were from absolutely shambolic defending by us.
"That has been a trait of ours all season, obviously, as well as not scoring enough goals - that's why we are bottom of the league.
"But the performance of the players in general was terrific."
The home fans among a crowd of 51,386 collectively held their breath after defender Tyrone Mears went down in the box under Charles N'Zogbia's challenge, although referee Andre Marriner was unmoved.
Jewell joked: "Tyrone said his footing went - it looked like a bit of a dive to me. I don't know why he dived.
"When I asked him at the end of the game, he said he lost his footing - so that tells me he probably dived."