In the past few years, superstars Robin van Persie and Cesc Fabregas, and other key players Samir Nasri, Alex Song and Gael Clichy have all departed the Emirates Stadium which has weakened Wenger's squad.
The Frenchman feels that a few years back, he had a group of players capable of winning trophies but has indicated that is not quite the case now.
"My regret is that we already had a great team two or three years ago which could compete on four fronts," he told Four Four Two magazine.
"We just missed out in the Champions League [last 16 tie] against Barcelona, when we could have scored in the last minute, and then we just missed out on the Premier League too.
"But you could feel the potential was there, and I thought, 'OK, let's do this together'. Then the team split up.
"Sometimes after five or six years' work, it is frustrating, you have to start all over again.
"We have lost recent players earlier in their careers - to lose van Persie, Fabregas, Nasri and Song in just two years, it is a massive amount of potential, of course you worry."