Jan Bednarek capitalised on some lax United defending on the 96th minute mark to flick on a corner for Michael Obafemi to fire home an equaliser.
Solskjaer told Sky Sports: "It was the worst time to concede a goal but it happens in football. We have won so many games in that way. It is all part of the learning for this team.
"You think you have three points in the bag but we probably didn't deserve three points today. Southampton are a good team, they run, and run and chase. We never got into our rhythm today. We played some fantastic football at times. The two goals we scored were very, very good.
"I don't think tiredness the case. It was just one of those days we did not take our chances and they took theirs. David de Gea had one really good save but apart from that we had the bigger chances but you are not going to be given three points and Southampton are a very good side.
"We defended well until that corner. It is disappointing but that's football. I know the feeling the other way as well."