Roberto Di Matteo's side take a 1-0 lead into the game. But after a controversial season, Ivanovic hinted expectation is lower this time around.
The Serbian defender, 28, said: "If people think like that, it can be good for us because we will feel less pressure, which in games like this can be very important.
"But Wednesday is a huge game for us. It's a big opportunity. We got a great result in Lisbon, but it's not over because they don't have anything to lose.
"They played very well in the first game so it'll be difficult. We have to be careful."