Manchester United took advantage of slips by Chelsea and Liverpool whilst Tottenham triumphed at Portsmouth and Arsenal enjoyed home advantage. Manchester United made hard work of taking over at the top of the Premiership after a scrappy 2-1 win over at Old Trafford.
United were two up by half time, an own goal from Zat Knight and Antonio Valencia's first goal for the club seemed to have set the Reds on the road to a comfortable victory.
But Matt Taylor's header with 15 minutes remaining set up a tense finale and Knight headed straight keeper Edwin van der Sar in the latter stages.
Aston Villa came from behind to inflict back-to-back away defeats on Chelsea with a 2-1 win at Villa Park.
Darren Bent's early fortunate strike proved enough to give Sunderland a 1-0 home win over Liverpool.
The Reds have now lost four times this season and the Black Cats leapfrog above them in the table.
There was a bizarre element to the winner as Bent's far-post shot deflected past Jose Reina via a beach ball thrown on to the pitch by a Liverpool fan.
Arsenal maintained their 100 per cent record at Emirates Stadium, but only after battling to overcome a spirited Birmingham performance.
Two goals in as many first half minutes put Arsenal in the ascendancy as Robin van Persie then Abou Diaby scored.
Lee Bowyer pulled a goal back before the break after a mistake by Gunners keeper Vito Mannone, but substitute Andrey Arshavin sealed the 3-1 win in the 84th minute.
Tottenham boss Harry Redknapp's return to Portsmouth saw him depart Fratton Park with all three points in a 2-1 win, as both teams ended with ten men.
The returning Ledley King headed Spurs in front just before Pompey old boy Defoe scored in first-half stoppage-time.
Former Spurs player Kevin-Prince Boateng pulled a goal back for Portsmouth on 59 minutes and, two minutes later, Defoe was sent off for treading on Aaron Mokoena.
In the final minute Michael Brown was dismissed after being shown a second yellow card.
Diniyar Bilyaletdinov's first goal for Everton rescued a point for the Toffees in a 1-1 home draw with Wolves.
The visitors took the lead on 76 minutes through Kevin Doyle, but Everton snatched a point two minutes from Bilyaletdinov.
Wolves striker Stefan Maierhofer was sent off for a second bookable offence after barging into Howard.
A James Beattie double was enough to earn Stoke City a 2-1 win over West Ham at the Britannia Stadium.
Beattie's first was a penalty after former Hammers winger Matt Etherington was fouled.
Matthew Upson headed the Hammers level before the break, but Beattie scrambled the winner on 69 minutes.