Central defender Clarke, 22, has joined for an undisclosed fee and signed a four-year-deal with the club.
"He's an imposing central defender and had a very impressive spell with Portsmouth, and has been a key player for them," said Potter.
"His challenge now is to compete with our existing centre-halves - an area where we have a lot of competition."
Brighton have lost right-back Bruno to retirement after the 2018-19 season, where they finished 17th in the Premier League.
Clarke said: "I'm over the moon to be here, these things always take a little bit of time, but I'm delighted that it's now been done.
"Everything here is set up for the club to keep progressing and that's one of the main things which attracted me here.
"The training ground and stadium are set up to achieve good things and the Premier League is where every footballer wants to be."