The supporters group are currently embroiled in a stoush with the club over ticket allocation. And McCarthy, who once played for the Eagles, noticed a big difference when his Saints team rolled over Palace in south London on Saturday.
"Is it less intimidating? Yeah, it is a lot quieter without them there," said McCarthy, who moved from Palace to St. Mary's two years ago.
"Obviously they are still noisy fans. But it is more noticeable without the drums there."
The club recently released a statement on the matter that provided little hope for a resolution.
It read: "Unless 100 supporters would like to give up their seats voluntarily, there is no real prospect of facilitating their wish of a move at this time.
"It is a great shame that they feel that they do not want to attend, but in the absence of any other suggestion it is difficult to see how we can do any more to make that happen."