Rafael was sent off in defeat to Chelsea after clattering into Luiz.
"Luiz has actually elbowed him twice," said Ferguson.
"It is quite clear on the camera.
"Whether the referee will do anything about it I doubt very much, Howard Webb won't be bothering himself about that.
"Rafael was certainly elbowed, but you know what happens when a player retaliates. He gets punished the most. That is what happened."