They are now seven points adrift of the Barclays Premier League safety zone after Christian Benteke's 17th goal of the season finally settled a see-saw game in Villa's favour.
Redknapp said: "If we had come in 4-0 up half-time it would have been about right wouldn't it? That's the best I've seen us play during that first half.
"It was so one-sided, it was unbelievable for an away team to dominate like that.
"But they got a lift from the equaliser after being absolutely awful first half and probably couldn't believe their luck. That goal knocked us for six.
"I'm upset for the lads. I thought they played so well today and we should have taken a point at 2-2 but then we gave a way a bad goal."
Redknapp is refusing to throw in the towel in the battle for survival.
He said: "We've got to keep going. We can't go home and feel sorry.
"It's going to be hard but there are still eight games to go. If we keep playing well, we can win four or five games.
"We're probably looking at four wins and two draws. It's going to be hard but it's not impossible.
"I'm not giving up. We've still got to keep going."