Caulker,19, was nicked and handed an £80 fixed-penalty fine for a public order offence in the Welsh city last month.
And the on-loan Tottenham centre-back, speaking for the first time about the incident, insists he will learn from his mistake.
Caulker said: "It was a tough time for me to deal with.
"I was arrested for swearing in the street and it was something I shouldn't have done and I've apologised for it to the club. It won't happen again.
"The manager had words with me and said I've got to walk away from situations like that.
"I'm still only 19 so I am still learning from these things. If I find myself in a similar situation next time I'll be wiser for it and not let myself get involved."