Giovinco scored in the 3-0 victory over Siena, but the supporters in Turin were not always fully behind the side.
"We should enjoy today's victory, as we had to win and it was not easy," Conte told Rai Sport.
"Siena are in great shape and they deserve compliments for their performance. We got three points and did very well in psychological terms to control the game.
"At times certain murmurs of complaint from the stands really irritate me, as I find them inopportune. People forget that 18 months ago Juve were seventh.
"They ought to be enthusiastic at having players like Giovinco, so I do not agree with those who jeer."