Poyet admitted he had expected Robbie Keane to convert the penalty against Arsenal as the Irishman is normally successful from the spot.
"Everybody was thinking about that," he told Sky Sports. "Unfortunately Robbie didn't score today but even after that we weren't expecting to lose the game."
Poyet also admitted Spurs were hugely deflated after the 2-1 defeat.
He added: "That is the problem, that is why we are a bit disappointed because it was a perfect game for us to get some points."