He was told 'Thanks, but no thanks' after trials with Watford, Barnsley, Swansea, Lyon and Auxerre.
He finally got his big break with French fourth division side FC Rouen, where 22 goals in 26 games earned him a move to Belgian side Mouscron before German club Hoffenheim paid €2.75million (£1.26m) for him in 2007.
"If there is pressure on me to keep West Ham up then I don't feel it because at the end of the day, football is a game," explains Ba to the Daily Mail. "There are far worse things in life than fighting relegation. There was more pressure on me when I was looking for a club.
"I didn't know where my life was going, that was real pressure. I've had so many trials in the past that didn't work out but I still wanted to make it as a footballer.
"I was between 16 and 19 at the time, but for me it wasn't tough. Many people maybe couldn't handle it but if you want something you have to go and get it.
"I don't fear rejection, I know what I'm able to do and knew if I didn't fight, I wouldn't make it. People were not going to give it to me for free."