Wenger initially called for Taylor to be banned for life from football, but later retracted the comment and admitted it was "excessive" and said "in the heat of the moment".
Kelly said: "I think Mr Wenger's (initial) comments after the game were so harsh. You know the way Martin is. He slid along the ground.
"He has gone in with one foot. You see players going in week in and week out, diving in with two feet, and lunging and that sort of thing is a disgraceful challenge.
"It is a terrible thing for Wenger to say, especially about someone like Martin."