Swansea meets Allardyce's Everton on Monday night.
Clement said: "We were so close to working with each other, but Bayern said 'no.' It was always only going to be a part-time role, I was not going to leave Bayern.
"I understood why Bayern said 'no'. They wanted me to concentrate solely on Bayern and rightly so with a club of that size. But I was disappointed because it would have been a great honour to work with the national team.