Following an impressive debut in a 3-0 win over Fulham, Carroll was forced off with a hamstring injury which led him to pull out of Roy Hodgson's squad for the back-to-back qualifiers against Moldova and Ukraine.
Allardyce believes Carroll will be an England mainstay in the future despite his absence on this occasion.
"We will just have to pray that it is not too severe," he said.
"It looks like a hamstring which is obviously the most disappointing thing from today.
"We were ready to bring him off and unfortunately on that last jump he got hit by (Brede) Hangeland and that twisted his body and pulled his hamstring by the looks of it.
"He is just so gutted because he has got injured and he can't go to England, that is why he is most disappointed because he wanted to get off there.
"We are going to play him and hopefully he will play every week for us and score goals and obviously Roy (Hodgson) is going to pick him - he wants to do well for us and he wants to do well for England."