Ferdinand, a team-mate of the Italian at West Ham, made the comment in his autobiography 'Rio: My Story'.
He said: "There were pictures taken of Paolo making what looked like fascist salutes which makes him an absolute idiot in my book."
Ferdinand revealed Di Canio could be difficult but was also hugely passionate about his football.
The Manchester United ace said: "He once told the Press I wasn't good enough to play for England. I pulled him about it and he said he had been misinterpreted.
"But I don't know, you were never quite sure with Paolo.
"For two weeks he would be one of the lads, then he would erupt and walk out of training because something wasn't going his way. He showed his passion more than any player I've known.
"Paolo got kicked in a practice game once and walked off. The manager Harry Redknapp pleaded with Paolo to come back but he wasn't having any of it. Paolo was a law unto himself.
"He's not the easiest person to work out but he was a top pro and a superb player."