Wayne Routledge and Jordan Obita had exchanged words after a free-kick in the dying seconds of the half, and when that argument became heated as they headed off the pitch others quickly became involved, including Fer and Martin Olsson from the Swansea side.
Eventually calm was restored but Clement revealed that Fer had been booked once the players had disappeared from the view of the crowd.
"I had gone down in the tunnel and then I heard a noise so I came back out and I do not know what had happened," he said after the Carabao Cup win.
"What I do know is that one of our players got booked. It was Leroy Fer, but I don't know what for.
"I don't think it had any impact on the game by the time you sit down and talk about the game it is all forgotten really."