The Premier League minnows, who were promoted in the summer, are struggling so far in the top flight.
Four successive losses is form that will see them relegated, but Brown says everyone in the squad believes they can turn around their season.
Per Luton Today, he said: “There is a massive belief in the changing room.
"Like we've just said in there, I think apart from the result, we couldn't have asked for much more than that.
“We've done everything we worked on during the week.
"If my chance goes in obviously it's a different scoreline, but yes, the belief is high and we've just got to go on to next week.
"We are progressing more and more and we can take confidence from that.
“The first three games before the international break were really tough and obviously it's a big step from last season, but we've been working really hard in training and you can see that in this performance.
"It's disappointing again, but we are progressing more and more each week and we can take confidence from that."