Villa fell two goals behind inside ten minutes but found themselves a goal ahead prior to conceding just before the half-time break which had the scoreline sitting at a remarkable 3-3.
Leandro Bacuna then drew Villa level when he scuffed home Matt Lowton's cross in the 25th minute and a stunning comeback was well on the way when the home side edged in front in the 37th minute.
Delph made up for his earlier effort when he controlled a header from Christian Benteke before smashing home off the underside of the bar but that joy was short-lived as Youssouf Mulumbu slotted through Guzan's legs after receiving Morrison's pass.
The second half, however, did not live up to the lofty expectations that the first 45 minutes provided although Villa fans were ecstatic when they were awarded a penalty just after the hour-mark after Diego Lugano wrestled Benteke to the ground.
The Belgian dusted himself off to beat Foster from 12 yards for his third goal in three games which saw Paul Lambert's Villa remain in tenth spot on the table with Pepe Mel's Baggies falling to 15th.
2013/14 Barclays Premier League season scores
Aston Villa: 4 (Andreas Weimann 12', Leandro Bacuna 25', Fabian Delph 37', Christian Benteke (p) 64') West Bromwich Albion: 3 (Chris Brunt 4', Fabian Delph (og) 9', Youssouf Mulumbu 43') Villa Park, Birmingham - 36,083
For breaking stories and all the great banter like us on Facebook: facebook.com/tribalfootball