The reigning Premier League champions booked their spot in the May 11 final, where they will meet Wigan Athletic at Wembley, by scoring either side of the half-time break today.
Samir Nasri and Sergio Aguero were on target before Demba Ba rescued one back for Chelsea but the 2011 winners managed to snuff out the FA Cup holders to be on the verge of more silverware which delights Kompany.
"It's great. You don't get to play a lot at Wembley and when you win it is historic for us," he said.
"Chelsea are a good team and you can see with the front players they have got you are always liable to concede chances.
"I think our strength is to be organised and to defend and we have been able to absorb it. They are very strong."
He added: "It is not a question of salvaging (the season). You never know how good your season is until the last game.
"We wanted to get trophies for our fans and bring back trophies to Manchester.
"If we can do it this season then we need to cherish it and we need to be happy about it because it is a big competition.
"You can always improve and we have still got a lot of the season to do that. We shouldn't look back too much now.
"I think anyone that participates in the FA Cup should take it seriously and that is what we have always done, not just today but in the previous rounds as well. It is a great competition."