Glen Johnson was left holding his head in his hands after seeing his late volley thwarted by a stunning stop from visiting goalkeeper Michel Vorm, while substitute Dirk Kuyt also had an effort ruled out for offside.
Downing told liverpoolfc.tv: "If Andy had scored with that early chance, it may have been a different game - Swansea might have had to change the way they played.
"It was a brilliant save from Glen's shot. I think the goalkeeper has probably been their best player this season, which shows how well he's done.
"But we had chances through Daniel (Agger), Glen as well as Andy's right at the start, and I'm not sure if Dirk's goal was offside or not.
"Swansea are a good footballing team and they've played well this season, but with no disrespect to them we need to be winning games like that at home if we want to get to where we want to be, so it was a bit disappointing for us not to get the three points."