Chelsea have failed with multiple offers for Schurrle this summer.
"André has been thinking about Chelsea - that's legitimate," said Voeller.
"Saying no was easy for us.
"It would be too soon for us to find a replacement of equal value. He and Lars Bender must stay here."