The Hoops produced their first-ever away win in the group stage of the Champions League thanks to Georgios Samaras' 90th minute header after Spartak's Emmanuel Emenike had scored twice to cancel out Gary Hooper's 12th minute opener.
A Dmitriy Kombarov own-goal allowed Celtic to equalise with just under 20 minutes remaining and then the Samaras finish came giving the Glasgow giants maximum points and delighting manager Lennon.
"I'm the proudest man in Europe. I can't speak highly enough of the players. They were magnificent, all of them," he said after the match.
"They thoroughly deserved this win.
"We didn't start the game particularly well, we looked a little bit nervous. Our passing wasn't there for the first ten or 15 minutes.
"But we scored a great goal through Gary Hooper, who I thought was world class tonight. People keep asking me about England and I think he showed everyone how good he is.
He added: "It's a huge step for the club psychologically.
"It's the first time we've won away in the Champions League so we have broken our record. But there is a lot of football to be played in the group.
"We are still novices in this tournament but a few more performances like that won't do us any harm."