The 70 year old reaffirmed his commitment to the Fir Park club on Wednesday following an initial approach from Aberdeen chairman Stewart Milne, but he performed a dramatic U-turn to leave Motherwell "surprised and disappointed".
Brown told BBC Scotland: "It's being a bit selfish but, when someone offers you a better job, I think it's understandable that we would be tempted.
"Impulsively, we said no because of the fantastic response from the staff, players and fans at Motherwell.
"But when asked to reconsider, we did. And I believe everyone has a right to change their mind."
Brown has swapped the challenge of securing Motherwell's fourth successive European adventure for an immediate fight at the bottom of the SPL.
But he said: "We know there are high expectations in Aberdeen and we have high expectations for every team we have anything to do with.
"We're not magicians, we're not saying we're going to do this or that, we're just going to work conscientiously and diligently."