Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2023-11-16

Channels

#xtdb
<
2023-11-16
>
zeitstein08:11:42

Is there a way to apply the same valid time to all ops of a single transaction (short of manually preparing each op)? I'm not seeing an opt for submit-tx and I don't think a tx fn could work.

zeitstein08:11:42

Manually preparing each op needs a lot of care, including modifying existing transaction functions, etc. Being able to pass valid time to submit-tx would be great.

refset10:11:09

Hey @U02E9K53C9L interesting question - it's not been asked before! Unfortunately there's nothing built in to help with that currently. I can see the value though, if only to reduce the amount of duplicate data being stored on the log :thinking_face:

gratitude 1
zeitstein11:11:23

Would this be a simple/quick change? If so, I might take a shot at implementing it.

refset14:11:26

It would probably be a simple change to make, but the implications require thought and discussion time which we don't have capacity for right now. Is working around it proving unexpectedly brittle/complex?

zeitstein15:11:12

No worries, I'll get creative with some guardrails 🙂

refset15:11:41

Cool, sorry I can't be more helpful with that. Happy to be rubber duck / sounding board in the meantime though!

gratitude 1
flefik11:11:34

Is there a good way to drop all the history before some timestamp, and just retain the latest version of every doc in the database? (trying to keep disk usage low)

tatut11:11:50

not to my knowledge, seems antithetical to what XTDB is meant to do

Christian Pekeler12:11:24

Migrate only the data you care about to a new instance of XTDB. We plan on doing that at some point.

✅ 1
🙏 1
refset14:11:37

longer term we are definitely going to be looking at requirements for configurable data retention lifecycle policies & automation (e.g. "evict certain classes of data after 7 years"), but currently yeah you really have to 'decant' into a fresh instance

👍 1
🙏 1
Filipe Silva23:11:22

How can I get the :xt/id from a tx on listen? I get something like this on deletes, but don't quite see how to go from that xtdb/id to something that gets me the id. The id here was "" in case it matters. 

{:xtdb.api/event-type
   :xtdb.api/indexed-tx
   :committed?       true
   :xtdb.api/tx-time #inst "2023-11-16T22:41:57.380-00:00"
   :xtdb.api/tx-id   1
   :xtdb.api/tx-ops  '([:xtdb.api/delete #xtdb/id c8d0e9aa0ad6ad1b22c4b232a822615a263d8099])}

Filipe Silva23:11:12

oh I really should have searched first 😓

Filipe Silva00:11:22

I had tried xt/entity-history before but was missing :with-corrections? :face_palm:

refset19:11:37

hey @UJVKWJTGE thanks for following up with the answer. Clearly this is something worth documenting 🙂 📝

Filipe Silva20:11:35

Well the listen docs on the site say it is experimental...

Filipe Silva20:11:07

I think the docstrings in the clojure lib no longer say that though, but not sure

refset20:11:09

ah right, yeah it's definitely not experimental nowadays, apologies for the confusion there

Filipe Silva21:11:40

Oh that's nice to hear, because I really on it a lot for a reason personal project I'm making

🙂 1