I guess the comparison will still have to happen in memory ultimately, but you can pass the data from the API into a query with :in and use something like [(xtdb.api/entity $ ?id) ?e][(= ?e ?data)]

So there is no way to have xtdb say run a checksum function against the document, and I can run the checksum against the data from the api, and just compare the checksum value?

Internally there are hashes that could do this, but I can't think of a clean way to access them via a query right now. I mean you could use internal APIs...but that's not ideal

The document hashes are visible in the open-tx-log API

Ok thanks, if this is doable, even if I'm using internal apis then I can scale my solution up to thousands of switches which is what I need.

I don't want to be writing the full document every 5 minutes 5000 times, when data only changed on some much smaller subset of switches.

Actually the entity-history API surfaces the hashes also, and is probably usable for this.

I'll look, just didn't want to pick xtdb if this was impossible. Thanks much for the info.

You can use xtdb.codec/new-id to generate your hashes and compare

Excellent, I'll get a small project setup and start playing. Thanks again and xtdb looks pretty amazing so far.

Note that open-entity-history will allow you to only retrieve the latest version

Happy to help :)