Walker, 22, is unconcerned about the stories that emerged from Chelsea after Villas-Boas was sacked earlier this year.
The England international defender went on: "Tottenham and Chelsea are different clubs with different players. I always judge things for myself and I have found it great working with him. There certainly hasn't been any bickering.
"I was eager to work with him after seeing what he did at Chelsea with the full-backs. He can take me to another level. Ashley Cole had a tremendous season. I speak to Ashley whenever we are with England and he hasn't had a bad word to say about him."