The Portuguese quit his job on Thursday amid the club's relegation to England's second tier this past season.
Silva took over the role in January from Mike Phelan, but despite his positive impressions, was unable to guide the club to safety. The 39-year old has since been linked with Crystal Palace, Watford and Porto.
And Allam agrees that these are sort of clubs that Silva wants to join.
"I had an expectation before seeing Marco yesterday [that he had decided to leave Hull]," Allam told Sky Sports.
"I'm very sad to lose him but I respect and understand why he has taken that decision. I understand why he wants to stay in the Premier League or got to a big European club with the prospect of European football.
"He's done an exceptional job. He has done fantastic for us but I accept he is a better calibre manager than the Championship."
Need a VPN? Or seeking to switch? Then support Tribal Football by clicking here to sign up with our official partners NordVPN. Not only will you receive the best VPN experience, but also great discounts by signing up via Tribal Football. Thanks for your support!