The Portuguese coach earned the reputation as a sporting showman during his entertaining three years at Chelsea.
But Mourinho told ESPN: "I love being a coach and the year I had without football after leaving Chelsea was difficult, but some parts of the job are not so attractive.
"Even though people say I like being in the media and being in the newspaper, the opposite is true. If I could avoid talking to the media from this moment on, I would do it. Still, I know football is like this. Players and managers have contractual obligations to talk and so we do it.
"I don't like being a star. I don't do this job to be famous or get my picture in the paper, but this is part of what we have to do.
"I don't like the fact that I lost my private life when I became a top manager. My family is paying for this and it affects my wife and my kids. My biggest job is making sure I am the best father to my children, but it can sometimes be difficult when you have this job.
“For me, the press conference before a game is the start of the next match as you can make some points that may help you when the whistle blows. You have to make the most of every situation."
Mourinho also branded as "fools" the critics who have wrote off Inter's Champions League chances.
"Everyone in Italy seems convinced that my team and every other Serie A side have no chance in Europe, but these people are fools who know nothing about the game. Some seem to just enjoy criticising and that's all they are good for. I ignore them.
"We are still a new team at Inter this season, but I feel we have a better balance with Samuel Eto'o at the front of the side. In my mind, we have made progress, but this can only be shown in the Champions League."