Evra, who previously described Pogba as like a "brother", claims the 26-year-old has been made a scapegoat for United's ongoing problems - even if he isn't playing.
"Paul, when he's playing, people blame Paul; when he's not playing, people blame Paul," said Evra on Sky Sports on Sunday after Roy Keane claimed Ole Gunnar Solskjaer should "get rid" of stars that were a "nuisance".
He added: "But I understand, sometimes I don't agree with Roy, but when you are a problem for the club... I give my example, when I kicked that fan in Marseille, the owner and the manager, they came and said: 'Patrice, what are we going to do now?'
"And I said: 'I'm leaving'. I don't want to bring any negativity around the club, and right now, maybe it will be time for Paul to leave, because I remember when it was a big game, Liverpool vs Manchester United, people were talking more about Jose Mourinho and Paul Pogba.
"This is insulting, for the two biggest clubs in the world to be talking about one player and one manager, that's why sometimes I think it's time, because when you play, they blame you, when you [don't] play, they blame you."