The England forward, who was a deadline-day signing from Tottenham, equalised in the 64th minute after Nicolas Anelka had opened the scoring 10 minutes after half-time.
But he felt he had three other chances to make it a dream debut on the south coast.
"We could have won it in the end, although I think that would have been a bit rough on Chelsea," said Defoe.
"I had another chance earlier and got tackled. Then I lifted my head and put one over the crossbar - but although you'll always regret things like that I'll take the one goal and I'm happy.
"I've come here to score goals and help the team get three points every week. Hopefully that's just the start but we showed today we have a lot of quality in the side.
"Chelsea had chances too but I thought we matched them all the way and that's really encouraging. They are a very good side even with their missing players."
"Jermain did well and took his chance. He'll score lots of goals for us and he could have had some more today but it was a great game and a good result for us," said Redknapp.
"We've taken a point off all the top four here this season and we're looking a strong side even without the four African boys (Papa Bouba Diop, Kanu, John Utaka and Sulley Muntari) who are still away.
"And Diarra is a top player. I thought his performance was absolutely fantastic - and that's three just like that in a row since he walked in the place a couple of weeks ago.
"Milan Baros worked his socks off as well for us.
Avram Grant, the former Portsmouth technical director who has presided over just two defeats in 31 matches since succeeding Jose Mourinho, said: "I thought we created the better chances but Portsmouth are a difficult side to face and they did well.
"Harry is very good at getting them going and they are strong side, but of course we are not so happy when we don't win.