The 25-year old Scottish left-back spent the second half of last season on loan at Pittodrie Stadium and the Dons boss is keen to have him back at the club but it seems that it may be too difficult following the Canaries' promotion to the Premier League.
Brown is aware that Smith is likely to get an improved deal at Carrow Road making it tough for Aberdeen to put forward an acceptable offer.
"I think it'll be quite difficult to get anyone whose, the rumours is that - wages double," said Brown.
"So quite clearly it will be harder to move him now than it was a year ago. I think the money might make it quite difficult for us."