Olympiakos started this season's Champions League campaign with a home defeat to Schalke - which Carroll believes makes Wednesday's game even more crucial for his side.
"We know what we can do. We did it last year. We lost our first game and bounced back - hopefully we can do that this year," he said.
"We have to go into these games to win. We have to come here and not lose the game.
"It's a big task coming to Arsenal but we know we've got good young talent and experience as well. It's a good mixture and hopefully we can pull it off."