Fork me on GitHub
#xtdb
<
2021-09-09
>
tatut04:09:38

as you deal with doc ids all the time, I'm liking the new :xt/id key

🙂 2
🙏 4
tatut10:09:11

I'll rephrase my question from yesterday: how do you know who deleted a document (which user)? there isn't a tx metadata afaict, should you just replace the doc with a new version that contains only information about the deletion instead of deleting the doc

refset10:09:40

Ah yes, sorry to have missed replying to this! Busy day 🙂 Your suggestion certainly isn't bad. You could also model dedicated tx metadata entity(s) that work more generically, getting updated alongside each transaction

tatut11:09:38

thanks, I didn't consider a separate metadata entity... that's one option

🙏 2
kevinmershon17:09:23

@U11SJ6Q0K that's what we're doing, we have a "transaction history" type object that stores the user, source microservice, transaction id, rationale, and a computed field-level diff

🙂 2
tatut04:09:36

interesting, so do you store a tx history document along with each tx you do? does the tx history document link to the documents that were touched in the tx

kevinmershon17:09:03

In a datalog query is there a shorthand for getting the valid-time/as-of timestamp of the document as part of the query?

kevinmershon17:09:35

Hoping for something like this in my when arguments:

[['doc :crux.tx/tx-time tx-time]
 `(<= ~start-timestamp tx-time)
 `(<= tx-time ~end-timestamp)]

kevinmershon17:09:13

Alternatively, can I somehow join to the transaction log by transaction id and get the time of the transaction in results for post-query filtering?

kevinmershon18:09:28

Another useful alternative would be, given a crux node and a transaction id, get the transaction object with the timestamp

refset18:09:41

Hey @U0D5Y2403 there's nothing built-in for this, but you can pull in the tx data like this (using the magic $ var which denotes the db context)

(defn get-valid-time [db e]
  (with-open [h (crux/open-entity-history db e :desc)]
    (:crux.db/valid-time (first (iterator-seq h)))))

(crux/q (crux/db (crux-node))
        '{:find [e vt]
          :limit 1
          :where [[e :crux.db/id]
                  [(my-ns/get-valid-time $ e) vt]]})
;;=> [[:bar #inst "2021-08-05T15:18:45.952-00:00"]]
(I had this pre-prepared here https://gist.github.com/refset/385d6dcfb66772aa1ec46fc20a49c633)

kevinmershon18:09:12

Woof, I don't have the entity on-hand, I'm trying to query ALL transactions as of a timestamp and get the latest id given a date

kevinmershon18:09:49

but maybe the magic $ will do it

kevinmershon19:09:15

I think we'll just hit the postgres table directly with

SELECT max(event_offset) as max_tx_id, min(event_offset) as min_tx_id
	FROM public.tx_events
	where tx_time between '2021-09-03T21:27:16-00:00' and now()

refset19:09:25

ahh my bad, I was too focused on the initial message 😅 > Another useful alternative would be, given a crux node and a transaction id, get the transaction object with the timestamp open-tx-log essentially does this, if I understand correctly - or have you dismissed that already?

refset19:09:53

the only caveat to using open-tx-log is that it doesn't use the node indexes, so it will hit the postgres table each time with something similar to the above anyway

kevinmershon19:09:37

interesting. If performance is an issue with open-tx-log I may go that route, but this is like 7ms currently

🙂 2
dorab23:09:47

In https://xtdb.com/reference/1.19.0-beta1/queries.html#custom-functions it says

Be aware during development that query compilation internalizes function definitions and this means that subsequent re-definitions of your custom functions may not be reflected until the query is modified and therefore re-compiled.
What is the recommended way to work around this during development?

refset23:09:29

One option is to add a dummy predicate clause like [(int? 1)] and then manually increment the value whenever you want to bypass the cache

dorab23:09:50

Thanks!

🙏 2