The win saw the promoted Hammers climb into sixth place in the Barclays Premier League table, to the delight of Allardyce, whose time as manager at St James' ended unceremoniously.
He said: "It's a fantastic start to the season for us now, 11 games, 18 points.
"Anybody who looks at what we have done, we had some reasonably good fixtures at the start of the season - every game is difficult in the Premier League, but they were more winnable than some of the fixtures we have perhaps got now.
"But in saying that, in our last three games, we lost to Wigan away, which I didn't expect, we drew with Manchester City and now we have beaten Newcastle.
"The big boys are coming up and we are still getting points, and that's the pleasing thing for me because confidence grows and grows and grows, as you could see today.
"It was a very professional performance by the players and nobody was overawed coming here."