The Three Lions were desperate for a win after a hapless 0-0 draw with Sweden in their first group match.
England have not made it through the group stage in the last three tournaments and it seemed as if history would be repeating itself after Slovakia broke the deadlock halfway through the first stanza.
However, Aidy Boothroyd's side showed spirit in the second half, finding the equalizer through Mawson's close range effort.
Shortly after, England were in the lead after a stunning counter attack resulted in a goal to Saints winger Redmond.
If England win their next match against hosts Poland, they are essentially guaranteed a spot through to the knock out stages.
Boothroyd, obviously delighted after the win, told Sky Sports: "I thought if we lost or drew the game we were out. So we had to be bold, keep ourselves in the mix.
"We did that. Now we've put this to bed we've got a really tough game coming up against Poland. We want to progress, we want to do well. I thought we deserved our win. We were the best team."