Aston Villa beat Brighton 3-2 but had to work hard to progress.
Nicky Forster netted Brighton's second in the dying seconds.
Accrington Stanley slipped to 3-1 home defeat to Fulham but more than matched their illustrious opponents.
Erik Nevland gave the Cottagers the lead but Stanley deservedly equalised before the break through in-demand striker Michael Symes.
However, Accrington were reduced to 10 men just before the break after Darran Kempson saw a second yellow.
Damien Duff and Zoltan Gera made certain of the visitors' place in the next round with two goals in the second half.
Birmingham's impressive form continues as in the all Premiership clash at Goodison Park they won 2-1 over Everton.
Christian Benitez and Barry Ferguson had the away side 2-0 up at the interval before Leon Osman pulled a goal back for the Toffees.
New Bolton boss Owen Coyle oversaw his first win as the Wanderers triumphed 2-0 over Championship Sheffield United at the Reebok Stadium.
Second-half goals from Gretar Steinsson and Johan Elmander were enough to see off the Blades.
Palace regained the lead early after the break when Ambrose scored direct from a free kick.
Ronald Zubar took the tie to a replay with an 84th minute equaliser.
Darren Bent put the visitor's in front after 15 minutes.
But three minutes before half-time John Utaka's clever back-header levelled matters.
Pompey then took the lead in the 57th minute as Utaka got his second and decisive goal.