The Foxes took the lead just past the hour mark via Shinji Okazaki after a cleared corner was hoisted back into the visitors' area and found its way to the substitute.
Islam Slimani then made sure of victory for Leicester with his stunning 78th-minute strike, breaking forward from a deep throw in and driving into the top corner to end Liverpool's hopes of progression in the tournament.
Klopp remarked afterwards: "The story of this game is really easy to explain: we were the much better team in the first half, we played good football – really good - didn't score, that's sometimes a part of football. We were unlucky in one or two moments, maybe not clinical enough in another moment. But as long as we concede goals like we conceded today again then it is quite difficult.
"We should have scored three, but as long as you leave games open, you have to defend. You always have to defend these situations better. [The first goal came from a] second ball after a corner, I think… it was close, but it was not offside. Really close, so we need to make it more clear it is offside and the only thing you can do is to push up. You cannot defend always like we did against Arsenal and [start] a counter-attack or whatever. We had to push up and we didn't. The second one is a throw in again and everyone can imagine we know how to defend them but obviously we don't do it.
"It doesn't feel too cool but it's the truth that we are still a good football team. I see it but unfortunately it doesn't feel it at the moment because we exited a cup competition. We wanted to go to the next round. Nobody was interested in who played the better first half, everyone was interested who will go through and we are not the team that will be in the next round for some good reasons."