Diego Simeone's side had to dig deep for the win, but the vital goal finally arrived in the second half through captain Koke.
Despite Rayo entering the game having failed to win any of their previous eight matches, they were never going to make it an easy game for their neighbours.
Atletico were reduced to just a couple of decent chances in the first half, with Joao Felix and Antoine Griezmann going close.
The breakthrough finally arrived just after the hour mark as Koke played a neat one-two with Joao Felix before passing the ball into the far corner beyond the outstretched Stole Dimitrievski.
Rayo never gave up and should've equalised on 78 minutes when Alvaro Garcia's cross was skewed over the bar by substitute Randy Nteka.
Atletico were reduced to ten men with five minutes to go as Angel Correa appeared to say something out of turn to the referee.
Nevertheless, Atletico held on for the win which sees them end the week in third position.
Need a VPN? Or seeking to switch? Then support Tribal Football by clicking here to sign up with our official partners NordVPN. Not only will you receive the best VPN experience, but also great discounts by signing up via Tribal Football. Thanks for your support!