Both clubs are currently in the relegation zone with 11 games left in the Premier League season.
But Pellegrino is adamant England's top knockout competition remains a priority for the south coast club.
"Yes we are taking the FA Cup seriously, for many reasons," said Pellegrino.
"Both teams have nothing to lose - we have to go forward because it's like a final. Obviously, we have another chance if it's a draw but that we have this possibility to progress is amazing.
"We want to move forward to the next round but also it can help us to improve as a team, to introduce players who have not played so much in recent games. It can also help us to believe more in our improvement since January.
"For those aspects I think it'll be really important to play seriously in the game, and to try to do our best."