The Blues have capped the attendance for Thursday's clash at their 41,631-seater Stamford Bridge stadium to just 2,700 in a bid to prevent violence between the clubs' notorious hooligans, reports The Sun.
Less than 400 fans watched their 7-0 home win over Wolves in the previous round at Aldershot's Recreation Ground - the venue for most of Chelsea's youth games.
The last-eight tie against the Lions was switched to Stamford Bridge on security advice.
Millwall requested more tickets after their allocation of 1,000 sold out in just a few hours.
Chelsea said no and decided to limit the number of sales to their own fans to just 1,400 - with the remaining 300 going to families, scouts, officials and media.
Need a VPN? Or seeking to switch? Then support Tribal Football by clicking here to sign up with our official partners NordVPN. Not only will you receive the best VPN experience, but also great discounts by signing up via Tribal Football. Thanks for your support!