Spurs fell 2-1, with Callum Wilson scoring West Ham's winner on 93 minutes.
On the chants of "You're getting sacked in the morning", Frank said in the aftermath: "Of course, I've probably had better times, it's probably not the best time of course. But I understand, I'm the man in charge. So the blame will go to me.
"That's fair, no problem in that sense. As long as they are backing the players, doing everything they can, to support them and drive them forward, that's what we do, and we will keep going forward."
The Dane also said: "If you're not winning enough, we know you will not get enough support from the fans. But when we're winning, it will change, when we start winning again. Which we will do. I'm not in doubt of that. I think also you can look a little bit at the performances.
"Let's say the Bournemouth and the Villa and this game here, if the players stop running, or stop doing anything, or not working hard, and we are not the team that's closest to winning, then you can say OK. But I think the team is working very hard. And I still think, and I know it sounds a little bit crazy, that we are close to something very good, than further away."
On doubts about his position, Frank admits everything appears to be running against them at the moment.
He said, "I think it also seems to be the perfect storm at the moment in many ways. We have a last-minute defeat. I think when everyone feels that everyone has given everything, including the fans, to back the team, and you get nothing out of it and you get a sucker punch in injury time, it's unbelievably tough to take.
"It's unbelievably tough to take for the fans, for the players, for me, everyone involved. And football is emotion, so I understand why it's tough. I can promise everyone it's extremely tough for me as well. So I think everyone involved in football knows this is just tough. But there's only one way. It's keep going. You can't feel sorry for yourself.
"Of course, I think probably tonight you're a little bit down, but then you should go again. There's only one way. That's going forward."