City won thanks to an injury-time strike from Matheus Nunes.
Tielemans said afterwards: "So frustrating to lose in the last minute of the game where we fought really hard - we blocked pretty much everything.
"To lose like that on that last action is so frustrating.
"We got the first chance and they got the last one. They score, we don't, that's football."
Top four hopes not over
Tielemans concedes their top four hopes have taken a blow with this result.
He continued, "We have four games in the Premier League left. We're going to give our all and hopefully we get a good result in and be back where we want to be in the Champions League positions.
"We have to analyse and see where we can be better.
"Saturday, we have the semi-final (in the FA Cup) which we can hopefully win.
"Now we have full focus on that. We are going to be having a tough night and then focusing on that."