The Red Devils have failed to win the title since Sir Alex Ferguson's last season in charge in 2013.
A major overhaul is expected this summer, with Ferdinand tipped as a candidate to takeover as sporting director.
Whatever the changes, the club legend wants to see United establish a clear pathway for the future.
"Man United are looking for their ideology still, he said.
"They went away from it after Sir Alex Ferguson, they tried to take it on under different managers and it didn't work, so they are still searching.
"There needs to be some sort of story at a football club for the stars and the staff to believe in.
"Do I think Man United can do that? I think time will tell - Ole has a window to work in and then he has to put that jigsaw in place.
"It is a big gap between Man City, Liverpool and Spurs. They will build again, they will recruit again."