Rabiot upset fans and Les Bleus coach Didier Deschamps by refusing to be one of the emergency reserves for France's World Cup squad in 2018.
Le Graet told L'Equipe: "It's more complicated if not every starter is playing regularly. But that is only my opinion. I also know Didier a little. He is not spiteful, but he has memory.
"I never had anything against him (Rabiot). I wanted to meet him, not to punish him. I am pleased to see that he is playing again in a prestigious club, where he has been in charge lately."