Lee has been accused of going "home to his mum and dad" after he refused a loan extension at Blackpool.
"Elliot picked up an injury and told us he didn't want to come back," Seasiders boss Neil McDonald said.
"He's gone back to the comforts of the Under-21's and being at home with his mum and dad. It disappoints me.
"We are trying to bring players here who want to play football. He's come and played all the games yet he isn't happy.
"He's gone back to West Ham and will probably be in their Under-21's for the rest of the season."