The French left-back scored his first goal since December 2014 as City produced a hard-fought victory over the Swans at the Liberty Stadium to move through to the fourth round of the cup competition they won last season.
Clichy opened the scoring before teenage midfielder Aleix Garcia added a second but there were some late nerves as Gylfi Sigurdsson scored in injury-time for the Swans.
"We could have avoided the goal as in England you never know until the final whistle. Overall we controlled the game but we didn't start well," Clichy told mancity.com.
"The first half wasn't good enough but then we started to play in the second half.
"The victory is the most important thing though because if you lose you are out of the competition so now we can rest and look forward to Saturday.
"It just happens sometimes there is a moment in the game where the game changes a little bit," added Clichy of his goal.
"The first half wasn't good enough but in the second half we came back with more desire so we have to keep playing like this."
City will now meet Swansea in the Premier League on the weekend.