Odds on Freedman succeeding the sacked Owen Coyle have plummeted in the last 24 hours.
"I cannot stop speculation, but I am very happy to be playing Millwall this weekend and that's all I want to say," Freedman told the Croydon Guardian.
"I cannot deal with speculation, I have too much to do in this job. I only deal in facts and the facts are that I have a game against Millwall this weekend and I am keeping my fingers crossed that my players come back from international duty without any injuries.
"I am also keeping my fingers crossed that I have prepared in the best possible way so that we win the match. Those are the facts, let's stick to the facts.
"When I was younger, 19 or so, and there was speculation that I'd be playing for Rangers and Celtic, at that age I found it quite difficult to deal with, but as you get older, you can brush it aside."