The Hammers dominated the first 60 minutes of the match and held a 1-0 lead thanks to Angelo Ogbonna.
But three goals in nine minutes from the Gunners saw West Ham extend their winless run at home to five matches.
"That pretty much sums up our 16 games," Rice told Sky Sports.
"First half we went 1-0 up and were frustrating them. Then second half they score two in six minutes. It is frustrating. We've missed a chance to move up the table.
"It is heartbreaking. We are in the rubbish at the moment. When one goes in we need to learn to keep our heads up. They have great players but they are all preventable goals. We need to look at ourselves. We are making the same mistakes. We are playing good football as well but making too many mistakes.
"We have to win our home games. It is horrible to hear the boos. We want to make the fans happy. But we have lost big games here. Now is the time to stay together. Games come thick and fast and we need to regroup.
"The manager is great, the players love the manager. We work hard all week and sometimes you wonder why we can't transfer that to the pitch."