He claimed a Lions steward had kicked a ball away as he tried to retrieve it during his team's 1-0 defeat in South London.
Diouf had to be restrained by another Millwall steward as he appeared bent on confronting the culprits.
And Millwall last night denied their fans had racially abused the Senegalese star after rumours on Twitter.
A spokesman said: "El Hadji has confirmed he did not receive any racial abuse and had no complaint to make."
However Police ejected fans after objects were thrown.