United conceded goals just before and just after half-time which killed the game for the Reds.
"To lose a derby game is horrible," Vidic told Sky Sports. "It's more difficult when you lose the game after this kind of performance.
"We were 4-0 down and it was really tough. We were never close to winning the game or even getting a draw. It's disappointing but we have to come back. We're down now but from tomorrow we'll try to be positive and win the next game and move on."