Villa made it four games unbeaten with a victory in the third round replay thanks to second-half goals from Ciaran Clark and Idrissa Gueye, prompting words of positivity from the Frenchman.
The Premier League's bottom-placed side have now produced two draws (including the initial cup fixture against Wycombe) and two wins in their past four outings which gives Garde a reason to believe ahead of the weekend's crucial league clash with West Midlands rivals West Bromwich Albion.
"I think we have the momentum still, yes," he said.
"Four games unbeaten now is very good but we have done nothing and won nothing, of course. But from the situation we were in the past few weeks, I prefer this one now.
"It is also important for the players to have scored two goals but we have to do better than tonight if we are to keep that momentum going."