The Irons kicked off the Premier League season poorly yesterday as they were thumped 3-0 at promoted Sunderland.
A section of the Hammers support are already questioning Potter's position.
But Brady told talkSPORT: "I wish yesterday could start all over again. It's so tough to take. It's never easy for the supporters, the players, or the manager to lose 3-0, particularly in the opening game of the season.
"I spent a lot of time with the manager and the squad in America on the preseason tour. The spirit amongst them is fantastic. I know that they'll be more disappointed, that they'll be the most disappointed people this morning.
"I know we'll see a reaction and I know they want to turn it around and they'll want to turn it around quickly.
"Graham is a forward thinking manager. He's sort of calm under pressure. He's very detailed in his approach. He's a great man, manager of players.
"He builds a special relationship with players and they understand where he's coming from.
"West Ham is not a club that panics about its managers. We tend to stick with them, tend to support people, stick with them and see it through.
"I really hope he does well. He's a pleasure to work with. He's incredibly professional. I know the players respect him.
"I know his relationship with the players is good. I know he'll be sitting down with them today to have a long, hard think about what went wrong yesterday, expecting a reaction and expecting to put it right."