The France defender is now interesting Tottenham and Barcelona after a superb World Cup.
Schindelmeiser told BILD: "Completing the transfer for Benji was one of the most difficult decisions I ever had to make in a responsible role because of the circumstances. And it was not a simple situation for Benji either.
"Jos Luhukay had told us shortly before the end of the transfer period that he did not want him and Carlos Mané."
Pavard would eventually join VfB for €5m - and can leave this summer for his €35m buyout clause.