The Dutch side deserved the slim victory as they outplayed Nuno Espirito Santo's side.
Vitesse left-sided midfielder Maximilian Wittek crisply converted a volley from the edge of the area proved the decisive goal of the game.
The result leaves Tottenham in third in their Europa Conference League group.
"It was a tough match against a good team," said Spurs boss Nuno.
"The Europa Conference League is like this; there are always tough moments.
"We have got two home games left and I truly believe they can help us solve the group."