Richards was given the option of being on England's stand-by list but turned down the offer – a decision which brought widespread criticism.
But Richards explained: "Once I wasn't in the Euro squad I thought there was a chance of me being in the Team GB squad.
"I was told if you are part of the England squad then you couldn't play in the Olympics. And I decided I wanted to be selected for Team GB.
"I didn't say I didn't want to play for England or anything like that. I just said I wanted to play for Team GB."
He added: "It was very disappointing for me not to make the Euro squad, especially after winning the Premier League.
"But that's what football is about, you can't dwell on the past – you just have to move forward. Now it's all about Team GB.
"It was down to the manager. Football is based on opinions, and all I can do is look at my own form, and my form last season was very good.
"I played in 40-odd games in a Premier League winning team – I would have thought that would have been enough, but it wasn't. In football, there's always ups and downs – you have to take them."