The Villa midfielder has come of age this season and has become one of the club's best players so there is sure to come a time when clubs with more money show their interest but the transfer talks are definitely an issue for O'Neill.
"It is irksome at this stage of the season," the Northern Irishman said.
"It doesn't do anyone any favours particularly because the player has had a fantastic time and is enjoying his football immensely here.
"He has had a really brilliant time, particularly since moving into the middle of midfield and I am delighted for him.
"James is still under contract with us and it's a case of dampening fires that weren't there in the first place.
"We want the best players to stay here at Aston Villa so we can keep the momentum going.
"But I have kind of accepted it now.