The Hammers had agreed a loan deal for the England international with a view to a permanent deal should the stay up in the Barclays Premier League this season.
But Carroll, who is also wanted by his former club Newcastle, was not prepared to leave Anfield for Upton Park.
"It was a terribly expensive deal with the loan fee and with his wages, which were enormous - too much for a club like ours," Sullivan told the Daily Mirror.
"But, as the manager really wanted him, we were prepared to pay it.
"Then there was a huge transfer fee which would have blown our budget for several years.
"My manager wanted Andy Carroll, so we backed him.
"Andy was the perfect player for him and the style of football we play.
"We play a lot of high balls, a lot of crosses - quite exciting football, not just humping it but clever crossing - and Sam wanted Jarvis to supply the crosses because he is a fantastic crosser.
"We had a deal in place with Liverpool to borrow the player which automatically became a purchase if we stayed up."