Potter knows his Brighton background will be held against him.
He smiled: "I'm sure it'll be a nice friendly one, definitely! I like the atmosphere there. A proper football stadium, supporters get behind the team, make it difficult for the away team. The boys are ready and itching to play.
On no wins in seven against Palace, he added: "It goes to show how tough Premier League is. Margins are small. That's history, looking forward to maybe changing that on Saturday."
Potter also says he doesn't want to be compared with predecessor Thomas Tuchel.
"Everybody has got slightly different idea on football. Don't get too wound up about that. It is more a choice we all make and we have to make those choices work. I have huge respect for Thomas and his teams. I'm not in the business of comparing myself to him or anyone else."