Klopp says Neville was right to apologise for saying he was "proud" of United's Hannibal Mejbri for his combative cameo at Anfield.
Mejbri came on as a substitute in the 84th minute of his side's 4-0 defeat on Tuesday. He was quickly booked for an overzealous tackle, almost cautioned again for another late attempt to win the ball and told to calm down by referee Martin Atkinson.
Klopp said: "He (Neville) had to apologise for that, rightly so. I understand where he is coming from, you want to see some aggression, but there is a difference between aggression and kicking players. That makes no sense.
"Aggression in football means you are ready to hurt yourself, not the other player. I don't blame the kid; he came on late and the game was quick and he wanted to make an impression.
"He was a bit late here and there, all good. Aggression in football is fine but you cannot sort your own problems by hurting opposition players. That is what I've never understood. I've played 325 professional games and without aggression and physicality I would have played none.
"It was the only strength I had, so I am completely fine with that. But it's about staying respectful, that's all. Things can happen in football, it is a high-speed game but not these kinds of things where you don't care about the other player."