The Black Cats have secured just two points from four games and are already being written off by some but Gardner says it is far too early to judge, especially after manager Steve Bruce bought in a whole heap of new players in the summer.
"We are four games into the league season - it's ridiculous," Gardner told The Journal.
"People are writing us off, but there's no way you can. I think people are jumping on the bandwagon.
"We haven't played that badly, we are not struggling. We are four games into the season, we have got two points and we have played Newcastle and Chelsea at home. We have actually had two good results away from home.
"I don't understand why people are saying we are struggling because, at the end of the day, we are not. We are playing well, we are just lacking a couple of goals, which will come.
"We are a top-quality team and there are four games gone. I can't believe it. The players we have brought in - look at them.
"There's experience, there's youth, there's pace, there's creativity. There's no way we are going to struggle at the bottom all season."