The 28-year-old goalkeeper, who won the Premier League with Hazard at Chelsea, left west London in 2018 to join Los Blancos for £35million on a six-year deal.
While his team-mate Hazard arrived at the club a year later after signing for £150million after scoring 110 goals in 353 games for the Blues.
Courtois revealed who the pair would discuss moving to the Spanish capital before they had completed their time at Chelsea.
"In the Chelsea dressing room, we were talking about the possibility of playing here," Courtois told El Mundo.
"'The White House, the White House...', we said.
"It was nice to speculate about it.
"Now, we are very eager to win trophies."