Mulumbu has not played for the Baggies since New Year's Day because of a knee problem and then being on African Nations Cup duty.
Clarke said ahead of tonight's game with Liverpool: "You miss all your good players when they are out or unavailable. Youssouf represented his country, we should have dealt with it.
"The players who have gone in there have done well for us, have tried hard for us, but it's always good to have a player like Youssouf around the place.
"He is infectious, he is enthusiastic, he wants to make things happen and, even having him back in training the last few days, has been good for the group."