The match, locked at 0-0, went into extra-time and was settled by Patrick Cutrone's 104th minute winner.
Gattuso declared: "This was a fundamental match. I must thank the guys.
"I won as a player, I'm young and I'm a beginner, I knew what I was up against by taking on AC Milan, we are struggling but I think that with work, the sense of belonging, we can get out of this.
"The team has quality and sometimes the ruthlessness has been lacking in this team. Today we have performed well, we have been able to fight and the quality has come out."
He added: "Now we have to get the most out of every guy, there's not much difference from today's game and the one against Atalanta, and now someone will take me as a dumb, but when we suffer a goal, we get into trouble."