McGinn had the visitors ahead with just over a minute gone and four minutes from the end of a one-side half Emilio Izaguirre grabbed the second.
Saints defender Danny Grainger, booked for dissent following the opening goal, was given a second yellow by referee Murray for a foul on Anthony Stokes before McGinn scored his second late-on.
"We've conceded five in the previous two games, so it was something we needed to concentrate on," said Lennon.
"The back four gave us a platform to go and win the game.
"All of them were superb, particularly Thomas Rogne, considering he's been out for a long time. He did well and the full-backs were excellent too."
"I'm not trying to send a message to anyone," said Lennon. "He (Loovens) probably hasn't done himself justice of late, and he's got competition.
"So we decided to change things round a bit, and Thomas came in and took his chance.
"We were delighted to get off to a good start - it gave us a platform to go and play in the second half, and if we had been a bit more clinical in front of goal, we could have won it a bit more comfortably."