The 19-year old has returned to Spurs following a brave goal-line clearance against Arsenal last Saturday which saw him make heavy contact with the post.
Caulker required surgery on the problem but is confident he can return a better player after experiencing Premiership football with the Swans.
"I'm devastated,'' Caulker told Swansea's official website. "It was my first time playing in the Premier League and I was really enjoying it.
"I was building up a good relationship with Ashley Williams and while we were still looking for our first win, to keep two clean sheets from the opening four games was very pleasing.
"I may have only played four games, but it was still a great experience which will only be of benefit to me in the long run.
"It's upsetting to pick up the injury now, but that's part and parcel of being a professional footballer.
"I've got to be mentally tough and make sure I come back stronger,'' he added. "I've really enjoyed my short time at Swansea and I want to come back when I'm fit and fight for my place again."