The Northern Ireland international was dismissed for a lunge on Cleveland Taylor three minutes after the break, with a Zander Diamond own goal eight minutes from time giving the Perth side victory.
McGhee said: "I really don't know what we have done to deserve this. The decisions that are going against us are just unbelievable.
"We can't give anymore than we gave in that game. That's all I ask of the players and that is all the supporters ask of them.
"The sending off has killed us as we have gone down to 10 men and lost a goal in a game that at half-time we felt confident we could go on and win. I had no complaints whatsoever except with the decisions that went against us."
McGhee described the red card shown to McArdle as "debatable", adding: "The sending off was the difference, I thought, between us winning the game and losing it.
"I thought Maguire gave him a short ball and he had to go for it. He had to protect himself a little bit and I don't think there was any malice in it. The referee said it was two-footed and I haven't seen it again.
"I don't think we would have lost that goal if we had 11 men on the park as we were a man down on the side that the goal came from. Eventually the 11 men found us out."
He added: "They are starting to feel that siege mentality, that things are going against them and the only way we are going to get out of this position is by ourselves.
"They came close today and I think if we had kept 11 men on the park we probably would have won that game.
"Today we went about our business the way I asked them. We are disappointed that we didn't take our chances but I was happy with the other aspects of the game."
St Johnstone chief Derek McInnes was delighted to see his side finally break their scoring duck after going four SPL games without a goal.
He said: "We said earlier in the week that we wouldn't care if it went in off somebody's backside. I felt last week we had turned a corner in terms of our performance as the players were excellent and were only undone by a decision.
"We have worked hard this week and I felt an added determination from the players to come out and play.
"We knew there was a lot of tension and anxiety with two teams both on a poor run but it is about keeping your head and doing the right thing. It may be a mistake that has won it for us but I felt the three points were merited."
McInnes thought referee Collum had made the correct decision in showing McArdle a red card.
He added: "We got the breaks today but I did think it was a red card as it was a two-footed challenge."