Taarabt has struggled this term and was left out of the squad altogether for the club's defeat at Norwich last week.
Warnock said: "He is like a few professionals in the game, especially people from abroad that have people around them who are alleged advisors.
"I said to Adel that if you were finishing next week you may lose one or two advisors. He's still growing up and it's hard for him when people put things in your mind.
"I have said to him that whatever happens, whether he stays or goes, he's got to work for Adel Taarabt, not for these people. It's not them who are going to make him a living."