With the Greek side winning 1-0 in normal time to force the extra period, Arsenal thought they had won it with a spectacular Pierre Emerick Aubameyang volley.
But with a minute of extra time to play El Arabi popped up to win the tie and put a huge dent in Mikel Arteta's plans for his first season at the club.
It was Pape Abou Cisse who had scored to make it 1-0 during the 90, with the centre-back heading home from a second half corner, but El Arabi was the hero as Olympiacos progress.
The game got off to a pretty slow start with both sides enjoying spells of possession without any real penetration. For the first 10 minutes there were no chances either way, with Arsenal appearing less aggressive in their press.
The first real chance came after some good tracking back from Pepe, who found Granit Xhaka on the edge of the box, only for the Swiss international's shot to be well blocked. Pepe then created space for himself inside the area, but his effort was also stopped by his defending man.
Perhaps the closest attack of the half came after 20 minutes, when an Arsenal break saw Alexandre Lacazette feed Pepe through the middle. The Ivory Coast international was tripped as he closed in on the box by Ousseynou Ba, who was lucky to get away with just a yellow, but his resulting effort from the free-kick was just over.
The next real chance of the half feel to Camara, who found himself in plenty of space on the left wing after Hector Bellerin and Shkodran Mustafi got caught upfield. However, Camara did not make the most of the opportunity and scuffed his left-footed shot into the side netting from edge of the Arsenal box.
With seven minutes to play in the half the Gunners had the ball in the net after a nice move involving Dani Ceballos and Bukayo Saka, who crossed for Lacazette to tap into an empty net. It was an excellent cross from the Englishman, but he was correctly flagged offside and the two teams went into the break with deadlock unbroken.
The second half continued in much the same vein as the first, with Arsenal dominating the ball but still lacking speed and accuracy in some of their passing. And the Gunners were quickly made to pay, when Pape Abou Cisse rose highest from an Olympiacos corner to make it 1-0. The set piece came from the Greek side's first foray into Arsenal territory of the half and it left the Gunners in need of an equaliser to avoid extra time.
Olympiacos nearly made it two minutes later after a great run from Kostas Tsimikas, who outstripped Bellerin and forced Bernd Leno into a good save with a fierce right-footed effort from the edge of the area.
Arsenal continued to press for a goal, but found creating openings difficult with players resorting to hopeful balls into the box. Olympiacos aerial superiority meant this produced little in the way of clear openings and with 20 minutes to play, Mikel Arteta brought Lucas Torreira on in midfield for Ceballos.
With Olympiacos reverting into a low block Arsenal struggled to make inroads, with their passing and switches of play too slow to cause issues. An audacious Pepe dribble nearly did the trick, but after turning defenders inside out on a couple of occasions, his shot was blocked for a corner.
Arsenal went close again minutes later when a Pepe shot was saved, with Torreira and then Lacazette unable to turn the ball home. Torreira had another effort minutes later when the ball spilled out on the edge of the area, but his shot was easily saved by Jose Sa in goal.
Olympiacos applied some pressure with a couple of late corners, but neither threatened and the game headed for extra time. Arteta refrained from making changes during the short break and with Joe Willock coming on late for Bellerin, Arsenal continued with a slightly lopsided three-at-the-back setup.
The home side continued to dominate possession and territory, but much like in normal time, Olympiacos' low block proved difficult to play through. A Pepe run nearly brought a foul on the edge of the box, but Arsenal's remonstrations were waived away before another corner came to nothing.
With minutes to play in the first period it was Olympiacos who had one of their best chances of the night, but Youssef El Arabi struck wastefully over from the corner of the box. It turned out to be the last effort of the half and Arteta readied Gabriel Martinelli for Lacazette as his final change. Olympiacos substitute Giorgos Masouras nearly put the tie beyond doubt at the start of the second period when his effort clipped the top of the bar after a sloppy giveaway from Torreira.
Aubameyang thought he had the final say, with a spectacular volley into the top corner six minutes from the end. The move started on the right, with a looping Mesut Ozil cross towards Martinelli, but his header was blocked and fell to the Arsenal captain who finished brilliantly with an over the shoulder volley.
But it was the Greek side's night, with El Arabi producing a sliding finish to win it late on. Arsenal defended the initial corner well but failed to get out properly for the second cross and it was the Olympiacos forward who got his toe to send the away fans into raptures.
The corner came after really poor play between Leno and Xhaka, with an under hit pass to the goalkeeper allowing the Olympiacos attacker to force a corner.
Arsenal v Olympiacos (1-2)
AFC: Leno; Bellerin, Luiz; Mustafi, Saka; Xhaka, Ceballos; Pepe, Ozil, Aubameyang; Lacazette
Subs: Martinez, Sokratis, Maitland-Niles, Guendouzi, Torreira, Willock, Martinelli
Olympiacos: Sa; Elabdellaoui, Ba, Cisse, Tsimikas; Bouchalakis, Camara, Guilherme; Randjelovic, El Arabi, Valbuena
Subs: Karargyris, Masouras, Lovera, Papadopoulos, Torosidis, Bullari, Gaspar
Need a VPN? Or seeking to switch? Then support Tribal Football by clicking here to sign up with our official partners NordVPN. Not only will you receive the best VPN experience, but also great discounts by signing up via Tribal Football. Thanks for your support!