The London club took the lead through Modibo Maiga, but a series of calamitous errors allowed the Latics to progress.
"We couldn't defend a fish supper tonight," the West Ham boss said.
"There were howling errors from our defenders in all four goals, it was comical.
"I never expected us to defend so poorly. It was really poor defending and all the good work we might have done with the ball meant nothing in the end because out of possession we were just rubbish.
It was an embarrassing defeat. They can't come knocking on my door about not playing now. They've got to take responsibility for their performance."