United won 2-0 at Crystal Palace on Saturday, but still face a huge task to clinch fourth spot and avoid missing out on the Champions League for the first time in 19 years.
"It's been a difficult season for United in the Premier League," said Carroll. "They used to score goals late on to win games, now they're conceding goals late on to lose games or drop points.
"That gives us belief; that even the best teams - like United - can be beaten. That's the way we're looking at it, with United not in the best of form at the moment.
"It's important for us to get something from the first leg to take to Old Trafford. We must get some sort of result at home first.
"I don't know why they've struggled so much in the Premier League, but we know how good they still are on their day and won't underestimate them. They won their Champions League group quite easily, but in the Premier League, if you make any kind of slip-up, it can lead to a goal against you, as we've seen with United this season.
"It's great for the Olympiacos fans to have United coming here. The situation over here [with Greece's failed economy] isn't great, but United coming over here keeps people's minds off it."
For breaking stories and all the great banter like us on Facebook: facebook.com/tribalfootball