With the game just minutes away from penalties, Angelo Ogbonna headed home to give the Hammers a 2-1 victory at Upton Park to break the hearts of the visiting Reds, following the 0-0 draw at Anfield in the initial fixture in late January.
The lead did not last long, however, as the returning Philippe Coutinho was clever when firing his free-kick under the Hammers wall which had anticipated an aerial ball and jumped straight over the Brazilian's well-placed set-piece.
The Belgian put wide a volley, had a one-on-one with Darren Randolph kept out by the impressive second-choice Hammers keeper and then headed over the bar.
As the match looked set for a shootout, West Ham won a dubious free-kick on the right flank which was whipped into the area by Dimitri Payet allowing Italian defender Ogbonna to rise above his marker and emphatically power his header beyond Simon Mignolet in the 121st minute.
It was a timely debut goal for the East London club from Ogbonna who has ultimately sent the Hammers through to the final 16 of the cup where they will meet Blackburn Rovers at Ewood Park on February 21.
2015/16 FA Cup - 4th round replay