Spurs new boy Roberto Soldado converted a 50th minute penalty after Dean Moxey was adjudged to have handled Aaron Lennon’s cross in the area.
However, Eagles boss Holloway felt his side should have been awarded a free-kick just before the penalty was awarded when Nacer Chadli brought down Stephen Dobbie and believes his team were harshly done by the referee and his assistants.
"If the referee didn't see that, the assistant should have. You have to ask if he's good enough if he didn't see that. I don't see how he can miss it.
"It's going to be a long hard season for me with these people (referees). I had this with Blackpool.
"Certain clubs get fouls and others don't. Tottenham will feel that if they play Man United.
"Do I think I would have got that at Tottenham? No I don't. I just want a bit of fairness.
"I want a foul like anyone else would have got one. I've got a horrible taste in my mouth after that."