Gillingham, who dominated the first half, reaped the rewards in the 19th minute through a Josh Parker header.
The Gills then went searching for a second goal, which they found in the second half, through an Ogilvie assist.
This goal proved to be the decisive goal, as even though Leyton Orient began a comeback late in the game, with a 79th minute goal from a low free kick, it wasn't enough.
Hear the thoughts of manager, Steve Lovell, on the match below:
Some brief thoughts from Caretaker Manager Steve Lovell & @Hessenthaler8_J on today's win v @leytonorientfc. #Gills@EmiratesFACup#FACuppic.twitter.com/pbWQ4TimDS
— Gillingham FC (@TheGillsFC) November 4, 2017
Gillingham now proceed to the second round of the FA cup, against an opponent that will be determined by the draw on Monday 6th November.