But the 24-year-old is loving life at Elland Road.
Speaking to FourFourTwo magazine, said: "I want to spend my career at Leeds and see the club doing well in the Premier League.
"I'm just enjoying every minute right now."
On an England call-up, Phillips added: "All I can do is stay fit and carry on doing what I'm doing.
"I know the bar is incredibly high for international footballers but I feel like I'm on the border of that.
"Once Leeds have become an established Premier League side again, I'll be competing with the best.
"Hopefully I can do well enough to convince Gareth Southgate that I deserve a spot in his squad."