Duffy was back in the starting line-up after Adam Webster was ruled out with an ankle injury.
He said: “It's disappointing. We didn't do ourselves justice in the first half, we didn't perform or play our normal game and that gave United the edge early on.
“We made some wrong decisions, gave the ball away in sloppy areas and they have fast players who can punish you. At 2-0 I had one chance when maybe I could have done a bit better but at half-time we regrouped and I thought we did better in the second half.
“We knew if we could get the next goal we could build from that. We've got some big lads and are a threat at set-pieces - we do a lot of work on them and we got another goal.
“But they scored again straight away and that was that. Their front three have a lot of pace so defending against them can be difficult - they are good players."