Coventry, backed by 11,000 fans, went ahead after 27 minutes when Michael Mifsud turned in Michael Doyle's cross.
And Mifsud killed off United with 20 minutes left, firing high into the net after good work from Jay Tab.
It was United's first defeat to lower league opposition on home soil since the 1995 humiliation by York City.
United had been the dominant side until Coventry grabbed their opener, yet Sky Blues goalkeeper Andy Marshall was not tested by the hosts who, while completely changed from the side that beat Chelsea on Sunday, still contained six full internationals.
One of them, Nani, a much-trumpeted £17million purchase from Sporting Lisbon, was a particular disappointment.
The Portugal winger got on the ball often enough but, for a player of his supposed quality, the final pass or shot was too poor to be true.