Spurs host Frank's former team on Saturday afternoon struggling for form at home.
"You need big b***s to manage Tottenham," Vega told the Daily Mail. "Thomas Frank is a politician, a diplomat, a yes man but he is not the person to say, 'this is me, this is my way'.
"That shows in the dressing room. When you are changing the players and the system every five minutes, players feel insecure especially when the crowd is negative. This is not a stable base. And I'm not sure he has the respect of the players.
"Tottenham is complex, the recruitment has been a disaster for years and now we are paying for it. And then comes a guy from Brentford, and it looks like the stage is too big. If they lose at home to Brentford he should resign. That would show me he has b***s."