Atalanta host the first-leg of their Champions League round 16 tie on Tuesday.
Rummenigge, a former Bayern chief exec, now is part of the club's supervisory board.
The former Inter Milan striker told La Gazzetta dello Sport: "We have to try to get a positive result in Bergamo and then try to reach the quarterfinals in the return leg. But it's not a given."
La Dea are Serie A's only team left in the competition.
Rummenigge continued: I wasn't expecting it. We shouldn't underestimate anyone this year. The so-called small teams are doing well.
"Atalanta is a small team that must be respected. In recent years, they've clearly won the final against Leverkusen, which was deserved in every sense.
"I repeat: it's not to be underestimated, but I also like the club's team. I met the Percassi family, I liked their perspective on football, how calmly they manage it."