After lacklustre draws against the USA and Algeria, he insists confidence has been restored in the England camp.
"I think we had lot of confidence coming in to the tournament," Milner told Villa's website. "We qualified very comfortably, played a lot of good football.
"In the first game, we played alright in spells, created a lot of chances and probably should have won the game but didn't. It maybe knocked the stuffing out of us.
"In the second game, we never really performed and then it was all down to the third game and to go and put in a performance like we did.
"We moved the ball around pretty well on a tricky pitch against a good team. To get that result when we needed it was a big boost.
"Hopefully we can keep improving. It is better to start the World Cup poorly and get better and better as you go on rather than the other way round.
"Hopefully that will be the case, we'll learn from the mistakes we've made so far in the tournament, move forward and hopefully improve as a team, then the performances will come and hopefully the results as well."