That's according to former LA Galaxy star Michael Ciani.
Ibrahimovic played alongside Cavani at Paris Saint-Germain from 2013 until 2016 before he joined Manchester United.
Following his time with Ibrahimovic in America, Ciani has lifted the lid on the relationship between the pair.
Speaking to RMC Sport, he said: "If you are close to Cavani, Ibra does not like it.
"Either you are with Ibra, or you are against him.
"He told me that everything was fine with Laurent Blanc at PSG.
"The only person he not get along with was Cavani.
"He told me that he only hated three or four people companions in his career... and one of them them was Cavani."