The Whites produced a two-goal lead thanks to Dimitar Berbatov and Sascha Riether but then conceded twice with Craig Gardner converting from the penalty spot and Stephane Sessegnon equalising on 70 minutes.
Australian keeper Schwarzer admitted it was a frustrating result as the Southwest Londoners had chances to extend their lead in the first half but in the end had to settle for a point as his opposite number Simon Mignolet saved the day for the Black Cats.
"It is definitely frustrating but the fact of the matter is, before the game we would have taken a point from this fixture, but it is disappointing. Even at 2-1 we had chances to make it 3-1 but unfortunately we didn't take them and we get caught on the counter but that's just the way it is sometimes," he told the club's official website.
"We knew what Sunderland's strengths were and how we could try and break them down once we did get possession. Generally we used the ball well in the first half and we did counter on them very well which is something we talked about before the game and it worked well."
On the save Mignolet made prior to Sessegnon's leveller, Schwarzer added: "That's the defining moment in the game and Mignolet did his job and did it very, very well.
"He's a very good goalkeeper who has played very well this season and they (Sunderland) have got him to thank for the point today. He made two very good saves in the second half in particular and had we scored those opportunities then the game is a completely different result but in the end we did get countered and we did concede."