Before then, Joel Matip had cancelled out Raheem Sterling's early opener for City for the two goals in normal time.
Sterling finally scored his first goal against his old club when he turned home David Silva's flick-on from close range.
The woodwork was hit three times in quick succession in the second half, as Sterling struck the post with a one on one, Virgil van Dijk headed onto the underside of the bar and Liverpool's Mohamed Salah hit a post.
The Reds were level when Matip headed home from Van Dijk's cross.
At that stage, Liverpool were the most likely team to win with Salah having at least three chances - most remarkably when his header was sensationally cleared off the line by a Kyle Walker bicycle kick.
The game went to penalties, with Claudio Bravo City's hero as he saved Wijnaldum's effort to make the ultimate difference.
City midfielder Kevin de Bruyne said afterwards: "It was a really tough game. You can still see it's early pre-season. Both teams are not physically ready yet. It's nice to see the competition and we will fight each other again this season.
"Some guys just came back two days ago, others have trained two or three weeks. Pre-season is more commercial these days so August is important."
Walker also remarked, "It's the first trophy you can win, for me growing up this was a big one, it starts off the season and hopefully we can replicate what we did last season.
"I'm a defender, I stop goals, I'm glad I can help the team out this time. It's too early to say, there's a lot of good teams in this league but hopefully we can be up there or thereabouts."