Martin was sacked and replaced by Ivan Juric by Saints in midseason and has since been appointed Rangers manager.
Solak admits losing sports director Jason Wilcox a year ago to Manchester United was also a blow before their Premier League return kicked off.
He told BBC Sport: "I really believe he was absolutely, fundamentally behind his idea of football and that is not easy in an environment like the Premier League that is so competitive.
"But to give him credit we were quite unlucky, I don't even know how to characterise some of the VAR decisions that we had - but in my opinion VAR took eight points off Russell's team."
Eight points more
Solak continued: "If he had those eight points more in December I think he would probably manage the team through the whole season.
"I'm not saying we would stay in the league but we would have a much more competitive run and Russell would definitely still be leading the team."
On Wilcox, Solak also stated: "I really liked working with Jason during his time here but I understand it, if you get the call from Manchester United, you go.
"We suddenly lost a man who started building something, we didn't replace him fast enough so we went into the challenge of the Premier League without a leading figure in our football department and that was very, very costly."