Jon Daly struck seven minutes from time with a powerful header as in-form United moved five points behind the Hoops.
"We are eight points clear (in third place) now. Yes, you look behind us, but you also look forward as well," he said.
"We'll have a wee pop at it, see what happens, take each game as it comes."
"I'm delighted with the three points. I knew it would be a hard-fought match," Houston added.
"I felt we coped quite well with what they had to throw at us. I thought defensively we were magnificent.
"We always thought St Johnstone would have a spell on top, it came in the second half, but our goalkeeper has given us a lift with the penalty save.
"We went up the park and put the ball in with a great goal from Jon Daly after the cross from Morgaro Gomis.
"I thought we merited the win."