Ferdinand was handed an eight-month ban and fined £50,000 for missing the routine test at the training ground in September 2003.
The former Manchester United captain passes the damning verdict in his autobiography The Second Half, which is released on Thursday.
In extracts released by the Daily Mirror, Keane wrote: "He (Rio) suffered for it and so did the team.
"If it had been me, and the doctor had said I had to do a drugs test, I'd have gone and done it. It wasn't something I'd have forgotten.
"It wouldn't have been like collecting a letter at the office, or remembering your boots.
"When a doctor says you've got to do a drugs test, it's not an everyday thing. But then, some people are genuinely forgetful.
"I don't think I was annoyed at the time, and I don't think the other players were either.
"But, ultimately, the team suffered. I didn't look at Rio and think that he'd been up to no good, or that there was a hidden reason for what had occurred.
"I think he genuinely forgot. We paid the price.
"He was a very good player and we missed him, especially in the second half of the season when the crunch games were coming up."
For the big stories and the banter follow us on Twitter: @tribalfootball