The League One club lost their place in the EFL on Tuesday when a takeover by C&N Sporting Risk fell through.
Bury-born Neville's mother Jill last week resigned as club secretary, while his late father Neville was a director.
"I'm devastated," said Neville. "It's disgraceful. Bury, after 125 years, no longer has a football club."
He added: "I find it difficult. I dread to think how my mum's feeling.
"I've obviously been living it as my mum's been secretary for 31 years. She resigned last week but still worked every hour that God sent to do the right thing for the football club.
"The heart of the town has been ripped apart. Now it's up to the Bury people - myself included - to try to put some heart back into a town that relied heavily on a football club."