Marco Silva's side have lost their last four outings and slipped into the relegation zone as a result.
Allardyce told talkSPORT: "Marco Silva's struggling because he hasn't recruited, it's the Director of Football. He's been brought in as a big reputation as Director of Football.
"I had a good relationship with Steve Walsh. At the end of the day, you spend all this money and if you don't spend it the right way it makes the manager's job harder.
"I wonder how many of these players Silva actually sanctioned to say they were the ones he wanted?