The 36-year-old - Germany's all-time leading goalscorer with 71 goals - left Bayern Munich in 2011 after struggling to satisfy the Dutchman.
He played 60 games while Van Gaal was in charge of Bayern but scored only 12 goals.
"It was a tough time, particularly working with Van Gaal," Klose told Die Zeit newspaper.
"I didn't feel free. It was very difficult for me to fulfil his expectations. He was asking me to make runs I just couldn't see.
"I gave it all I could, but sometimes it just wasn't enough."
For the big stories and the banter follow us on Twitter: @tribalfootball