The fullback was given a straight late on into the 2-1 win after lashing out at Joao Pedro.
Having already been sent off once this season at Newcastle in November, his three-game ban has been upgraded.
Advertisement
Advertisement
It means James will miss Sunday's finale against Bournemouth and the first three games of the new season.
Blues manager Mauricio Pochettino said: It's difficult to give my opinion because I didn't have time to see the action. But it's painful (for us) because it's a fantastic night of football and he's a really important player.
Now he is going to miss the next game and a few games next season."