Pressure has eased on Conte after victory over West Brom.
But Gullit warned in The Sun: "If there's a confrontation with the coach and it hasn't been ended by the board, the players think 'ah [we can say what we want]', especially the players that don't play.
"All of a sudden your authority is gone. It happens because nobody at the club is saying it's not true.
"If they were to say 'we love Conte, we're going to stick with him', nobody could say that.
"But what's happened with all the previous coaches is that no one (at the club) says that.
"All the players, especially those that don't play, think 'he won't be here next year, let's wait for a new one', where they will get a better chance and will play. For a coach it's the worst thing that can happen."