The loss leaves the Tigers just three points clear of the drop zone and it was an opportunity missed according to Brown.
"The players showed a lot to try and get back into the game," he told Sky Sports News.
"Obviously the goal we conceded was a poor goal from our point of view. We gave the ball away on the edge of our box and they think the goal was offside.
"Some days you get them but I don't think it was offside but some days you will get them.
"You know we have had 13 shots at goal against Aston Villa, 13 chances to score and we haven't.
"Give them a bit of credit, they have thrown bodies at the ball and blocked it. That's the second time they have come away 1-0 winners and not deserved it."