Arsenal also had the chances to win the game earlier, with Cottrell wasting perhaps their best opening, but the Gunners now progress to the fifth round of the FA Cup for the third consecutive year.
Southampton got off to the better start of the two sides and were nearly ahead within a minute when Watts hit the post.
A ball in behind Arsenal right-back Joel Lopez was met by Marko Rus, whose cross found the onrushing Watts, but the resulting shot cannoned off Hubert Graczyk's left post.
However, the Saints' supremacy didn't last long and Arsenal quickly grew into the game with Cottrell beginning to set the tempo in the middle of the park.
Nathan Butler-Oyedeji was a key source of attack for the Gunners and on 16 minutes he found space on the right hand side, but Alfie Matthews couldn't quite meet his dangerous cross along the face of goal.
Keogh's header was intended for Bycroft who had come off his line, but the ball was far too firm for the keeper and only just ran beyond the chasing Greenwood.
Five minutes before half time Southampton were forced to replace left-back James Morris with Zuriel Yaiwo, after Morris pulled up earlier in the half after being bypassed by the impressive Butler-Oyedeji.
Arsenal remained on top at the start of the second half, but the first real chance fell to Olaigbe, following a good run from Jankewitz, whose shot from the edge of the box was comfortably handled by Gracayk.
Cottrell and Greenwood then failed to test Bycroft from presentable set piece opportunities, before Alex Kirk headed over from a Cottrell corner.
On 67 minutes the Saints had a penalty shout when Ramelo Mitchell went rushing through on goal, only to go over under pressure from Kirk, but the referee waved away appeals from the Southampton players.
With the Saints looking for a winner, Lucas Defise replaced Caleb Watts in the number ten role and he nearly had an instant impact as he almost beat Graczyk to a dangerous through ball.
The Gunners were next to make a change with the dangerous Butler-Oyedeji replaced by Dennis, and Greenwood switching across to the right wing.
Moments later and Dennis should have put Arsenal ahead blazing over the rebound from a saved Cottrell effort.
Cottrell was initially put through by Greenwood but had his left footed effort saved from 10 yards out by Bycroft, only for Dennis to miss the rebound.
However, Dennis made up for the error with ten minutes to play, breaking the deadlock from the edge of the box with a powerful drive following a mazy run across the pitch.
Bycroft was beaten at his near post, but such was the ferocity of the shot he could hardly be blamed.
Southampton did have a late opportunity to equalise but their attacker Rus just strayed offside, consigning the Saints to what will have been a disappointing defeat.
Arsenal: Graczyk, Oyegoke, Lopez, Patino, Kirk, Ogungbo, Butler-Oyedeji, Cottrell (C), Matthews, Cirjan, Greenwood
Substitutes: Monlouis, Mitchell, Plange, Dennis, Laing, Sraha, Flores
Southampton: Bycroft, Keogh, Morris, Chauke, Kpohomouh (C), Bailey, Rus, Jankewitz, Mitchell, Watts, Olaigbe
Substitutes: Wright, Tchaptchet, Defise, Turner, Watts, Smith, Yaiwo
Need a VPN? Or seeking to switch? Then support Tribal Football by clicking here to sign up with our official partners NordVPN. Not only will you receive the best VPN experience, but also great discounts by signing up via Tribal Football. Thanks for your support!