Goals from Greek internationals Giannis Goumas, Dimitris Papadopoulos and Dimitris Salpigidis - two of which came from free-kicks - inflicted a defeat which leaves Aberdeen rock bottom of Group B ahead of their next game against Lokomotiv Moscow at home in two weeks.
"It's just a shame to have worked so hard to get here and go down in the way we did with those two free-kicks.
"I know that Panathinaikos are a well known European club but we thought we had a real chance of getting something from this match. In the end they didn't have to work hard for the victory."
Despite the result, Calderwood is confident his players will get the club's European campaign back on track with their home game against Lokomotiv.
"I'm sure the boys will bounce back from this," he said.
"We know how important the home games are and my players want to their best irrespective of what's happened tonight."