The powerful American striker was unlucky not to open his account in the Black Cats' eventual 3-1 loss to Arsenal at the Stadium of Light on Saturday after Martin Atkinson failed to play advantage during an incident in the second half.
Altidore had outmuscled Gunners right-back Bacary Sagna just outside the area, with Atkinson calling a foul on the Frenchman, but the Sunderland attacker played on without hearing the whistle and managed to score past Wojciech Szczesny which was disallowed leaving the former AZ Alkmaar man flat.
"Of course I thought that was my first goal in the Premier League. You don't get chances like that in this league very often and I thought I had my goal," he told the Sunderland Echo.
"It was very disappointing. I didn't hear the whistle or anything. I don't think Sagna heard anything because he kept going and I think it was (Kieran) Gibbs who tried to get it out of the goal so he obviously didn't hear anything either.
"The keeper came out and played the ball like it was live.
"It crossed the line okay as well, so it's disappointing."