The Foxes were handed a 2-1 defeat by Watford on Boxing Day after conceding from two set-pieces and following the disappointment, Maguire wants the Foxes to improve from dead ball situations in order to avoiding the concession of cheap goals.
Maguire said: "We are all very disappointed. I don't think it was a bad performance and we controlled the game. Then we conceded from two set-plays, so that is disappointing from our point of view.
"The performance itself wasn't too bad and there were some positive aspects but there were some areas where we need to improve and certainly defending set-plays is something we need to nail down. We have conceded too many this season.
"First and foremost we are not one of the biggest sides. There is me and Wes Morgan who like to head the ball and there is the fact we don't play with a big striker.
"We also missed Vicente (Iborra) today who is another big player for us in the box, but no question we have to be better in the box. We are working hard on that on the training pitch."