Havertz's close-range finish handed the Gunners an early lead at Anfield, and it proved to be the winner on Merseyside, as Mikel Arteta's team ended a 14-year wait for three points at the stadium.
Arsenal had to dig in during the closing stages to get the result over the line and Havertz thanked Rice for his cute assist which was vital at the final whistle.
"This means a lot to the whole football club, and it's been 14 years, that's a crazy stat. I think we are all happy we gave it to them today.
"We all know what this stadium is about and what their fans are about. Liverpool have big players and they can show their quality. We knew what was coming for us and we stuck together and defended together.
"90% of the goal belongs to Declan Rice."
A win for Arteta means Arsenal jump back above Manchester City at the top of the table with a three-point lead on Enzo Maresca's charges.