League One Leyton Orient have submitted an official bid to the Football League to move to the London Olympic stadium after the 2012 games.
But Orient chairman Barry Hearn has confirmed the club will withdraw if they cannot reach an agreement over the athletics track.
Hearn told the club's official website: "We're asking for a 25,000-seat stadium and we want to see if we can get around the athletics track. It has to stay, we know that. But can we build up, if not down, and see if it's possible to get it covered while we play?
"If it isn't possible, it isn't - and we wouldn't move in with a running track between the pitch and the crowd.
"We have submitted an application for permission to move to the Football League and that permission is something that West Ham don't yet have. It was the Premier League who gave them the green light to move in their original plan, but they're no longer members of the Premier League.
"The rules are that the Football League would not sanction a club moving closer to another club if it affects the business of that other club, and there's no doubt that West Ham moving there would affect us."