Ferdinand, who painted his face with a St George's cross for luck, watched the last-16 clash with family and friends while recuperating from his knee injury.
Pundits back home savaged England's defending, but Rio admitted to The Sun: "I think all areas of the team could have done better.
"The first goal you'd be disappointed to concede at any point to be honest.
"It's one lofted ball from the keeper straight through and within two seconds the lad's in on goal and scored.
"We'd be disappointed if we conceded that ever again.
"I'm sure the lads will be hurting now but they'll be honest enough to say, especially with that first goal, that they're disappointed with conceding it."