On-loan centre-forward Romelu Lukaku is set to return to Chelsea.
Defender Olsson said: “Romelu has been massive for us. It’s not realistic to think we keep him.
“Someone who scores 13 goals for a club like West Brom, with all respect, it’s a great achievement.
“He’ll be difficult to replace but that’s the challenge.”