England No1 Hart, 26, is under the spotlight after a string of high-profile errors has left his Manchester City place under threat.
Milner said: "He's confident enough. He knows what comes with being England's No.1. With Manchester City and England you get to the top and then you're there to be shot down. People talk less about the good things you do because they're expected and more about the mistakes you make.
"He's mentally strong enough to deal with that, that's not a problem, but we're all human beings and if the players around him can give him a gee-up, even if he doesn't need it, that will help.
"He's a victim of his own success because of how good he's been and how quickly he's become England's No.1 and playing for Manchester City. People said he had a poorer season last season but he won the Golden Glove.
"Some of his performances and the saves he made, the Dortmund game stands out. We could have lost that game 8-1 very, very easily and that's not exaggerating. It was one of the best goalkeeping performances I've ever seen."
For the big stories and the banter follow us on Twitter: @tribalfootball