The Hoops now lead Rangers at the top of the SPL by three points.
Scott McDonald opened the scoring for the visitors but United striker Francisco Sandaza scored twice at the start of the second half before a Naylor volleyed saved the day for Gordon Strachan's men.
The Celtic boss enjoyed the game, saying: "It was terrific, absolutely terrific.
"I think the players and fans enjoyed it.
"If anyone can find a negative - and I'm sure they will - then they are warped.
"I thought we would get the equaliser, we put enough pressure on although I didn't expect it to come from Lee, that's for sure.
"Their goalie has made a couple of important saves and as important as Sandaza was in scoring, the keeper was terrific.
"In the first half we played well, they played well for 20 minutes in the second half.
"We weren't professional at the start of the second half and we paid for that and then we came good for the last 25 minutes and I thought, 'Here we go again, we are going to win this game.'
"There's been no change in the title race.
"At this moment in time, the two teams are not good enough to take a grip of it."
United boss Craig Levein stated, his side on their day are a match for both Old Firm clubs.
"I thought it was a great game, it was end to end and both teams had chances," he said.
"Anyone who watched the game would have been happy with what they got for their money.
"We don't fear playing either of the Old Firm.
"We always think we can win and the players believe that.