Pardew's much-changed team - he fielded only two of the men who started Saturday's 1-1 Barclays Premier League draw at Aston Villa - took the lead on four occasions at the City Ground, but were pegged back three times as their npower Championship hosts made them fight all the way to the whistle.
Skipper Fabricio Coloccini eventually sealed the win in injury time at the end of extra-time to leave his manager dreaming of an extended run in the competition.
Pardew said: "It's a competition where we will be looked at as one of the possible teams who can win it, and we have got to believe that.
"We are looking for a home draw because we have had two tough away games.
"The first game might have been unglamorous, but to win here at the City Ground is always a good result, so hopefully [we will get] a home draw."