The 25-year old defender was brilliant for the Tigers last season, prompting rival clubs to show interest, but Brown hopes he can hang on to him although he believes it is inevitable there will be a move to a bigger club somewhere down the track.
"I am very keen to hang onto him. There is no doubt about it, he was our best player last year," Brown told Sky Sports News.
"But there are some big clubs out there that can turn young players' heads quite easily.
"I will be fighting tooth and nail to keep him but everyone has their valuation.
"Hopefully nobody comes up to our valuation and we can hold onto him for a year or two."