Vidic went on the stadium PA to tell supporters to calm down during Saturday's match in Klagenfurt, Austria.
Vidic, 28, who was standing in as skipper for Dejan Stankovic, also promised an improvement in the Serbs' final World Cup warm-up games with Poland and Cameroon.
He said: "Fans have been phenomenal. I'm sorry we've not responded with a win.
"But there is no reason for concern because there is enough time to get everything in place.
"We have worked hard to qualify for the World Cup and believe the next two friendly matches will be better games.
"But all these are preparation matches - our main goal is South Africa."