Cristiano Ronaldo, fresh from scoring five in the 9-1 rout of Granada over the weekend, broke the deadlock at Campo de Futbol de Vallecas midway through the second half for his 300th Real goal before James Rodriguez netted soon after.
That kept Real four points behind Barca, who had beaten Almeria earlier on Wednesday.
Real coach Carlo Ancelotti said: "We rushed things too much in the first half. I told the players they had to apply pressure with more intensity up front in the second half and in the end we played a very complete game and deserved our win. It was more a case of us increasing our intensity levels than Rayo's dropping off. We were a lot more at ease in the second half.
"I leave here very happy and am hugely satisfied with the second half performance because our attitude was good, we played well and we were physically strong. That's all I'm looking for."