The Welsh midfielder went down in the area under pressure from Swans defender Dwight Tiendalli when the game was scoreless during the second half but referee Howard Webb waved away the protests because he felt the Gunners man tripped over his own heels.
"It was a penalty and the referee saw it," he said.
"Do you know why he did not give it? Because he thought the guy had not done it on purpose, and I also think it was accidental.
"But he (Ramsey) did not trip himself, he was caught clearly by the leg of the Swansea player. The referee hesitated and he did not give it."