Joe Cole, on debut, had set up a double for defender James Collins with superb crosses from the left flank in each half to turn the tie around after Tom Cleverley had fired United into the lead on 23 minutes.
However, as the clocked ticked into stoppage time, Dutchman Van Persie - who had started on the bench - ghosted into the right side of the penalty area following a superb crossfield pass from fellow substitute Ryan Giggs, before drilling a low shot into the bottom corner to force a replay.
Ferguson said on MUTV: "We missed some good chances, with headers over the bar and (Shinji) Kagawa had one blocked on the line, I thought it was not going to be our day.
"But you have to say the pass from Ryan Giggs, then the first touch from the ball from Robin and his finish was world class."
Ferguson was delighted to have made sure they did not suffer a disappointing early FA Cup exit.
The United boss continued: "We want to stay in it and have a go at it this year.
"We have had some terrible draws over the years and been knocked out early and we don't enjoy that.
"For us to be losing 2-1 in that game is amazing with the way we played.
"But when you play West Ham you have got to defend those crosses and set pieces, because they are brilliant at that, the best team in the league, but we did not do that.
"Twice in the same position, (it was) James Collins from a cross by Joe Cole and we have lost two goals.
"I could say it is a travesty, but they are good at it, but with some of the chances we missed, we should have been home safely."