The 21-year-old looks set to leave White Hart Lane this summer after dropping down the pecking order under Juande Ramos.
Kaboul's representative, Rudy Raba, has stated the defender will stay in England, but will not be making the move to Sunderland.
"Younes will definitely stay in England, but not at Tottenham," Raba told Sky Sports News.
"The manager is not fair with him. The manager doesn't want him.
"Roy Keane was a fantastic player and he is a very good manager.
"Younes has a lot of respect for him, but why would he go to Sunderland?
"He doesn't want to go there."