Roberto De Zerbi was heavily backed with over £300 million worth of new signings in the summer, but Spurs are still without a Premier League win after five games.
Tottenham fell to a 3-2 defeat to Aston Villa, going 3-0 down before managing to pull two goals back, their first of the 2026-27 campaign.
The North London club will now head into the September/October international break firmly in the relegation zone.
Speaking to TNT Sports, Van de Ven said: "I think in the first half we played really well and created a lot of chances. We conceded with 10 men, and that is always tough.
"The second half is just not good enough from us. The way we conceded goals today is not good. I want to review it first, but there are things that are not acceptable, and we have to do better.
"We need to stay positive. It is difficult right now because everyone is going on the international break, but when we come back, we have to work."