The midfielder is on-loan at Chelsea this season.
Kovacic said: “Well, after two years of struggling in Madrid, playing and not playing, it was difficult.
“You don't have confidence, you don't try things you know. This year was my final decision to go.
“[Luka Modric] obviously was not happy that I would leave because we had three amazing years together winning a lot of trophies.
“This year I looked only at myself and this is what I need. He obviously was happy for me that I got what I wanted."