Allardyce refused to be drawn on the matter in the immediate aftermath of the defeat as he said he had not heard the chants.
Today he defended his decision to not speak out sooner.
"I was doing a press conference after the game and had no idea what had happened and what had been said or done," the 58-year-old said.
"I wasn't prepared and didn't want to comment having not heard what went on.
"I don't know how I was supposed to react to something I didn't know anything about.
"You are in a very difficult position. We had just been beaten 3-1 by Tottenham. I wasn't in the best mood. I am not expecting the question.
"I am expecting to talk about football and I am a football manager. I didn't want to comment on it. Now, like everyone else - I don't condone it."