Ogilvie, 21, has yet to make a first team appearance for Spurs and spent most of the last two seasons on loan at League Two Stevenage.
The former under-16 and under-17 England international made 18 appearances for the Boro last season.
"I'm looking forward to it and I want to be playing games regularly," Ogilvie told the club website.
"I'm really pleased to secure Connor's signature. He's got pedigree having come through the youth ranks at Tottenham and what is an added bonus for us is that he has experience of playing league football," added Gillingham manager Ady Pennock.