Reports have claimed 21-year-old could be recalled by Chelsea and then loaned out to a Premier League club, with Wolves particuarly interested in signing the England international.
Given Abraham has scored 16 goals for Villa this season, Smith insists the Blues are happy with the striker's progress.
"He's enjoying himself as a person and developing as a player," said Smith. "All three parties (Abraham, Chelsea and Villa) seem very happy with what's going on.
"There will always be speculation but I'll be amazed if he, Chelsea or ourselves would want to upset that."