The on-loan Chelsea man reportedly headbutted Villa teammate Jota, with his future at the club now being discussed.
Phillips says he's surprised about the conduct of his former Leicester teammate, but admits Drinkwater has always had a temper.
Speaking on talkSPORT, Phillips said: "We've all been in training grounds. Scuffles happen and you keep it in house.
"But unfortunately for Aston Villa, when things are going against you and you're fighting relegation, these things come out.
"Knowing Danny, he has got a short fuse. There have been other incidents as well on and off the pitch.
"But this surprises me. It wasn't that long ago he was playing for England. It's just incredible how the player has gone downhill.
"Danny played a huge part in Leicester's title-winning side and went for big money to Chelsea; there was criticism from all over whether it was right or wrong, but he couldn't turn Chelsea down at the time.
"But since he's been there he just hasn't played. Nine [sic, ten] starts in three years?
"Whether there's an underlying problem away from football we don't know, but it's just sad to see from a player of his quality."