Referees' union Prospect are insistent that Chelsea should apologise to Mark Clattenburg and also pay him compensation.
Prospect national secretary Alan Leighton said: "Chelsea have done a number of things wrong with this case.
"Firstly, they made the allegation and the nature of it public on the Sunday of the match.
"Secondly, on the Wednesday after their detailed investigation they decided to go ahead with a formal complaint based on one single witness statement.
"The FA statement makes it clear Chelsea didn't even know when the alleged comment was supposed to have been made during the game. No organisation in the world would say that was ever going to get past the barrier of evidence needed to bring a charge.
"Thirdly, they have not apologised to Mark, who has gone through hell. This is about an individual who has been badly wronged and there is not even recognition about what they have done to this referee and that is staggering."
However, Leighton says Prospect do not intend to take Chelsea to court.
"That is not our intention at this moment," he added. "Our intention is for something to happen that avoids any need for legal action.
"We have been calling for Chelsea to make an apology and pay compensation because it's the right thing to do."
Leighton says clubs need to be prevented from targeting officials unless there is clear evidence of wrongdoing.
He said: "There needs to be some form of deterrent to stop clubs going ahead with stuff like this, which has no basis and impacts so badly on the referee.
"Our concern is it could happen from any club and is more likely to happen again if there is not some acceptance by Chelsea that this was wrong and shouldn't have happened."