The Blues have capped the attendance for Thursday's clash at their 41,631-seater Stamford Bridge stadium to just 2,700 in a bid to prevent violence between the clubs' notorious hooligans, reports The Sun.
Less than 400 fans watched their 7-0 home win over Wolves in the previous round at Aldershot's Recreation Ground - the venue for most of Chelsea's youth games.
The last-eight tie against the Lions was switched to Stamford Bridge on security advice.
Millwall requested more tickets after their allocation of 1,000 sold out in just a few hours.
Chelsea said no and decided to limit the number of sales to their own fans to just 1,400 - with the remaining 300 going to families, scouts, officials and media.