Williams, 22, spent much of the past year on loan at Wycombe Wanderers, where he helped the club gain promotion from League Two last season.
England Under-19 left-back Brown moved to the Terriers on New Year's Day from Tottenham Hotspur.
Exeter City boss Matt Taylor hopes Brown's versatility can allow him to be more flexible with his line-ups.
"He is a dynamic left-sided defender. We have got the experience of Craig Woodman and Dean Moxey who can cover that position, but this lad is going to get up and down that touchline time and time again," he said.
"It gives me an option if I wanted to play five at the back because he can play wing back."
Need a VPN? Or seeking to switch? Then support Tribal Football by clicking here to sign up with our official partners NordVPN. Not only will you receive the best VPN experience, but also great discounts by signing up via Tribal Football. Thanks for your support!