Steve Pressley's men found themselves with a two-goal lead at the Falkirk Community Stadium thanks to Farid El Alagui's 58th and 73rd minute goals before the Scottish champions struck back in the final eight minutes through Dorin Goian and Nikica Jelavic.
And just as the third round match was headed for extra-time, former Celtic youngster Mark Millar sent his free-kick goalward to see 'Gers keeper Neil Alexander let it slip through his fingers and into the back of the net.
This sparked dramatic scenes for Falkirk who will move through to the next round of the cup competition and after the game, Pressley was full of pride for his Bairns side.
"I feel remarkably proud of my players," he said.
"They showed a great desire and determination and they deserve all the praise that's going to come their way.
"I look at the game and I look at the gulf between the sides.
"It is a remarkable achievement and I believe these lads will go on to great things.
"But I don't think they will achieve many better results in their careers.
"Their mentality was incredible and they showed true belief in themselves."