Liverpool looked like they had thrown away two points when Johan Berg Gudmundsson's 88th-minute equaliser cancelled out a stunning strike from Sadio Mane. But Ragnar Klavan grabbed the winner in injury time.
“I'm frustrated, obviously with the final moments of the game because we delivered a good performance and deserved a point," said Clarets boss Dyche.
“Sometimes though, these things happen. It's a soft goal but it happens and not very often to us.
“You have to remember that's virtually the same team I played the other day. There's a lot of noise about not being able to do that, but everyone saw today that was a highly motivated group of players going hard to win a game.
“The margins of football are unbelievably tight. We were a bit unfortunate at Manchester United, then dominated at Huddersfield and had a clear penalty not given, and unfortunate again today.
“But there has to be a performance level and I've been delighted with that over this Christmas period, especially when we are at stretch at the moment with the squad as thin as it's been for a while."
Dyche added: “We've done everything we can to get something out of it against a Liverpool team that, towards the end, are taking time over throw ins and free kicks and killing the game and wasting time.
“That's a marker in itself; to have a team like that resort to that against Burnley. That hasn't always been the case, trust me!
“There was a lot made of them being without a few players today, but I looked at four of their front five and they were all brought in for over £30m.
“That tells me they are still a fine side, make no mistake, but I felt we kept them quiet for long periods today.
“So, although there is a frustration in the final few minutes, there's also a lot of pride in how the team has moved forwards from a year ago.
“We have to continue to work on being a threat in the final third, but there's huge progress been made in this squad."