Faris Haroun was the goal-scoring hero for Belgium with a 35th header, but it was Chilean referee Pablo Pozo who ultimately decided New Zealand's fate with a harsh red card to Daniel Ellensohn just after the break.
"I saw some of the FIFA officials after the game who said it was a bit harsh," Oly-Whites coach Stu Jacobs said, "but if you look across the tournament a lot of the decisions have been mighty harsh so I guess there's some consistency there."
"That doesn't make it any easier to play with ten men, though.
"You can't fault the effort of the players when they're a man down, they were absolutely fantastic again.
"The referee was perhaps a wee bit inconsistent in the way he let some things go and pulled up others for us.
"But that's just part of the game and you just have to get on with it, and I think we did that."
New Zealand finished 14th in the 16-team field behind Australia, Serbia and China on goal difference with Jacobs giving his side a mark of 7/10 in their first Olympic campaign.
"We fell short of the goal we had set internally, which was to make the quarter-finals, but we didn't fall short by much."