Hammers defender Angelo Ogbonna scored a header in the 121st minute to send the London club into the fifth round of the cup in what was a massive night for the East London club.
"Tonight, obviously, you couldn't write it," captain Noble told BT Sport.
"We had a similar situation last year with Everton but it went to penalties.
"To be honest, I am out there thinking 'where am I going to stick my penalty?' Then Angelo steps up and saves everyone the heartache of penalties, even their players.
"Do you know what? This is the FA Cup, innit," he added.
"Extra-time, two minutes to go and Angelo pops up with an unbelievable header. I mean, it's a great ball from Dimi and we've got so much height in there. We don't score enough from set-plays.
"What a time to do it. The crowd are waiting for the winner and there's no better time for it."