The Hammers were able to deny the Saints for the entire 90 minutes in difficult conditions on England's south coast, with goalkeeper Jussi Jasskelainen the star of the show, pulling off a string of fine saves.
However, Allardyce felt his side deserved to have a man advantage after Southampton midfielder Morgan Schneiderlin produced a poor tackle on Mohamed Diame in the first half which did not even bring about a free-kick from referee Andre Marriner.
"We feel a bit disappointed that Schneiderlin is still on the pitch - we didn't even get a free kick - because it's a two-footed challenge," the Hammers boss told Sky Sports 1.
"(With) two-footed challenges, we have been told by everyone in the referees' department from Mike Riley down, there is no other choice but a straight red - this one we didn't even get a free kick.
"I can't for the life of me [understand] - Andre (Marriner) is such an experienced referee now, how he didn't see that.
"For me that was a disappointment. It's in the end kept them with 11 men on the field when the law was supposed to be they should only have had 10 men which would have given us a better chance of winning.
"But I'm happy with a point. I can't affect what the referees do, it's not my job."
The draw was West Ham's second of the season and sees them sitting 10th with five points from four outings as Southampton find themselves on the same amount of points but below the East Londoners by virtue of goal difference.