The Blues have delayed plans to rebuild the 41,000-seat stadium. And Sidwell believes the west London club don't receive the same support as their Premier League rivals.
"Some stadiums get up for the big games more than others," Sidwell told Jim White on talkSPORT.
"But generally, unless there's real entertainment on the pitch, it's a quiet stadium.
"That's from playing there and going as an opposition player as well."