The Scot was offered a coaching role at Stamford Bridge three years ago.
He told the Lancashire Telegraph: "I had only just gone to Coventry and I felt the timing was just wrong.
"The chairman Ray Ranson had given Chris Coleman and me the opportunity and, at first Ray said I couldn't go.
"Then Chris Coleman stepped in and said 'If you want to go, go for it'. But the timing was not right for me, I felt I couldn't leave so soon after joining.
"It would have been a great move and a great experience, of course it would. I don't know how long it would have lasted though, Scolari didn't last that long.
"There was also another dynamic in that Steve Clarke had signed again in the interim period before Scolari signed. If I was going to go as assistant manager, Steve had already signed a contract as assistant manager.
"There would have been two of us standing there and the new manager would probably have wanted me to have taken the sessions so it would not have been right."