It marks a return for the veteran, who played for Rayo on-loan early in his career with Atletico.
Marca says the signing of Costa was carried out by President Raul Martin Presa, much like last year's addition of Radamel Falcao, which wasn't met with approval by other club officials.
Costa has thus put an end to weeks of uncertainty over whether he would return to Rayo.
The 33-year-old will sign a one-year contract plus an option for another, and will earn 1.5 million gross per season, which could increase if certain targets are met.
The striker joins after nearly eight months of inactivity, with his last match dating back to December 12, 2021 when he helped Atletico Mineiro lift the Copa Do Brasil.
Since then, Costa has been training in Madrid ahead of his next move.