The Blades almost held onto a draw with Liverpool in Sheffield before the international break, but a late goal from Gini Wijnaldum sealed a 1-0 victory for the visitors.
They didn't make the same mistake on Monday, with Lys Mousset's first-half goal proving the difference against the Gunners.
Speaking after the game, Wilder said: "To host a game like this, the magnitude, to play like that and get the win. We limited them to few chances and backed up a good performance with a good result. We did not get that a few weeks ago against the European champions but three points is a great reward for the players.
"It is absolutely crazy that we can open up against Arsenal. we saw their ability, skipping round our challenges so we had to balance it in and out of possession. Getting ahead gave us something to hang on to. Dean Henderson made a couple of saves but it has tipped from the Liverpool game where we should have got something and tonight we did and we deserved it."