The former United defender faces today in the Capital One Cup final.
O'Shea, 32, said: "The history there can't be as big as United until they keep winning things and maintaining success.
"That's going to be the key for Man City. United will always have that history and will always have that success because it's been proven. The structure that's at United should always guarantee they're going to be very close to success.
"But City will be challenging for these honours continuously in the next few years because of the quality they have.
"Clearly, with the financial strength they gained, they've been clever about the way they've improved things there all round.
"They can only succeed in terms of the structure they have and I think most would agree they have the strongest squad in the Premier League. Watching City grow, you were kind of thinking - like Chelsea beforehand - it's going to happen, they're going to be successful at some point because they have to succeed.
"City like to think they're the Manchester team. United have a bit more of a global branding and that's always going to be the case. That's the banter among the fans."
For breaking stories and all the great banter like us on Facebook: facebook.com/tribalfootball