Hey @U8ZN5EHGU, hope all is well! Thanks for the kind words ☺️ > Is it at least once delivery? Assuming the node isn't restarted it should be exactly once. > Do you receive the whole log or only from the moment you start listening on? Only from the moment you start listening. We have talked internally about somehow combining open-tx-log and listen as a single API, but not made any real steps towards it. > And does anything around this change with XTv2? We haven't built an equivalent API yet. In SQL land we may implement something like LISTEN / NOTIFY around the built-in log table - or we could even look at upcoming standards https://www.confluent.io/en-gb/events/current-2022/sql-extensions-to-support-streaming-data/. Whatever we end up doing, we'll definitely also ensure parity via Datalog. > But what if you messed up and a bug generated way too many documents or worse, it generated a ton of changes for the same document. How do you clean up the mess? Presumably the real impact/concern is about the cost of handling this data ~forever after? If it's really too expensive, then the best thing to do is to decant the data into a fresh XT instance with all the erroneous data filtered data. > does this work any differently for XTv2? The model is more or less the same, but v2 will make it much less expensive to keep the erroneous data around ~forever, so hopefully you can avoid ever thinking about decanting and be satisfied with a :delete

Thanks Jeremy for the detailed answer! All is well and I have a new chapter coming up in my professional life which I am definitely looking into XT for 🙂 I must admit I forgot about open-tx-log. Together these two functions are definitely enough to build reliable log followers. Looking forward to seeing how the V2 version will look like! You are right, that with a bitemporal system the main concern of having excess data in the log is cost. It is much less scary than I am used to from other systems. And great to hear that this is even less of a concern with v2!

Awesome. Looking forward to hearing what you're up to 😁