Despite creating several chances, the Blues were frustrated by goalkeeper Antonin Kinsky.
Mike Penders' own goal handed the visitors the lead before Cole Palmer's dismissal compounded Chelsea's misery.
Mathys Tel sealed victory late on, leaving Alonso disappointed with a performance that failed to match recent wins over Bournemouth and Everton.
“We all take the blame for the result and for the performance,” Alonso said.
“Today we were not at our best. I felt that in the second half, the game was even and could have gone either way. Once we conceded that first goal, and then with the red card, it was difficult.
“In the second half, it was tight, with some half-chances for them and half-chances for us. It was about taking a moment and once we conceded the first goal, it conceded everything. Then, when we wanted to have a reaction, we got the red card. Then it went the other way.
“We can do better. It's one to reflect after this big game that it was against a good opponent. And the performance, it's one to improve.”