The German giants stood off Saul Niguez allowing the Spanish youngster to dance past three opponents before slotting coolly past Manuel Neuer in the 11th minute of the Champions League semi-final first leg.
In the wake of the 1-0 defeat, Guardiola criticised his team for allowing the goal to occur in the manner it did in what was a subpar opening 45 minutes from Bayern.
"Their goal is a brilliant goal, but it's our mistake," said the Spaniard.
"We kept standing off. I'm not happy with how we played.
"It was a good game overall, but we started badly. The goal was a consequence of our slow play."
Bayern dominated large portions of the second half but were unable to breach Jan Oblak's goal as Atletico drew first blood in the final four of Europe's premier club competition ahead of the return bout at the Allianz Arena next Tuesday.