Terry was banned for four matches and fined £220,000 by the FA, but escaped criminal punishment over the incident.
Ferdinand and Terry were centre-back partners at international level, and the Manchester United legend wishes he'd spoken about the incident at the time.
Quizzd in an interview with The Beautiful Game Podcast if his relationship with Terry had improved, Ferdinand said: "No. The John Terry and my brother's situation, if I could rewind the clock back now, I would have spoken out from day one and said to my brother, 'speak now'.
"We were advised, you have to remember, there were lawyers in the background, representing the club, the League, the PFA, the FA, so you had all these different people talking saying what's right, what's best for you, and you get in this cloud, this bubble, you just take on the professional advice.
"You don't want to be selfish, and end up being quiet in the background hoping the situation plays out how it should. The facts are there for people to see.
"The problem is, we did it from a pure heart. Our hearts are pure in this situation, but not everyone is like that. When your heart is pure and you don't speak out, you get punished. I've worked that out now.
"You're the ones that get punished and tarnished. You're the ones that get labelled. Whether that's because you don't speak or because you're black, I don't know. I just think now, I would speak straightaway. I'd tell him [Anton], speak now."