Many fans believed that Bilic made the wrong decision in allowing Payet to leave in the January transfer window, especially as he was a fan favorite at the club.
However, Bilic claims he had no choice, given Payet's determination to leave and his impact on the dressing room.
"With Dimitri he had a great season," the former West Ham boss said.
"He enjoyed it and was brilliant on the pitch, off the pitch and in the dressing room. He was vocal in a good way, having fun and mixing with all the players.
"In the second season he was doing everything right on the pitch but he just wanted to go. He had some private problems and wanted to go.
"Of course, at that moment I tried to persuade him to stay because he was our best player. He was very liked by the other players and giving us that something extra.
"The club wanted him to stay, I got that message from the board as well so we were on the same wavelength and we wanted to improve. We didn't want to lose our best players.
"I tried to persuade him [to stay] and spoke to him a few times but when it came to the point that it started to damage the dressing room and the players started asking what was going on, then I had to react.
"I spoke with the board and unfortunately you have to draw the line."