United put Sunday's 4-1 derby loss to Manchester City behind them by beating arch enemies Liverpool 1-0 in the third round at Old Trafford on Wednesday night and Evans, making his first start of the campaign, said the league cup is a target for the reigning Premier League champions.
"We want to win it. It was the first trophy I won and it was a great feeling to win it then," he told Sky Sports 1 after Javier Hernandez scored the only goal of the game.
"It's a competition we want to win, we will be taking it seriously."
Evans added: "I think the fans enjoyed it, we certainly enjoyed it.
"It was a hard game, Liverpool have been playing expansive football.
"I am just glad we got the goal after half-time and held on.
"They put out a strong team. There were a lot of lads out there who haven't had a lot of football and were getting a bit tired towards the end."