Olsson has a year left on his current contract, while Andrews is on a short-term deal to June.
"It's very hard when a player is dealing with his contract but Jonas is a big player and we need him," said Mulumbu in the Express & Star.
"He is the kind of player every player wants. He shouts on the pitch and brings people with him and that's the type of player we need in the team.
"He is a very big player for West Brom and I hope he will speak with the chairman and Dan Ashworth so they can find the right situation for him."
On Andrews, he added: "I think experience could be his biggest strength because when you play with him you feel a confidence."