Palace winger Townsend said: "We were undone by a bit of quality at the end and everyone is devastated in that dressing room.
"We know the fans are going to create an incredible atmosphere and we wanted to start fast to keep them on our side.
"We did that and again in the second half then for 15 minutes it went a big ragged and the game got away from us.
"We knew they had quality in the starting 11 and off the bench but obviously we are gutted. Maybe we should have shut up shop at 2-1."
He continued: "We were still going strong in the last 10 minutes. David de Gea pulled off a world class save and then one of their players pulls off a world class strike at the other end.
"It's the Premier League and unfortunately we were on the wrong end of it tonight. We have proved we can pull off shocks against the top sides.
"We will put on a show on Saturday [against Chelsea] and hopefully pull off another shock."