Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2023-08-03

Channels

#xtdb
<
2023-08-03
>
Jenny Kwan00:08:41

This seems like a basic question: how do I generate an UUID that is guaranteed to be a new document :xt/id?

Jenny Kwan00:08:21

I suspect there's no built-in way? So if I want to ensure new ids, I'll have to do it in a transaction function? Which then raises the question of how to return the generated id from the transaction? Given that multiple operations may be batched in a single transaction, how would I go about getting any returned values for the submitted transaction?

refset08:08:32

Oh, it's actually really important that you don't do anything non-deterministic, like generating UUIDs, within transaction functions. XT needs to be able to consistently replay the function and get the exact same result. Therefore you really need to generate the UUIDs externally and pass them in as parameters to the function.

Jenny Kwan00:08:19

I know this is overly paranoid, but collisions do happen. And separate from that, the question of how to return values from transactions, including transactions that include transaction function calls, still stands.

refset08:08:05

> but collisions do happen Sadly ID / UUID collisions can't be detected automatically, but at least you should have a history of conflicts to unpick things... I guess you could add a second UUID into the mix if you were really paranoid πŸ™‚

refset08:08:09

> the question of how to return values from transactions, including transactions that include transaction function calls in short you can't do this directly, due to the asynchronous processing - instead you have to store the return values under some known ID that you can then query for

marcofiset17:08:18

You could use cuid2 for collision resistant IDs https://github.com/hden/cuid2

Jenny Kwan00:08:29

Finally, how do I abort a transaction from within a transaction function call?

refset08:08:33

instead of returning a list of ops, you can return false

Jenny Kwan00:08:49

Oh, one more thing, does ::xt/match match only the attributes on the supplied document revision, i.e. if the stored document has attributes not specified in the document revision, will the transaction continue or abort?

refset09:08:49

The match is an exact match, so the set of keys and corresponding values must be the same. So in your case the transaction will abort

Jenny Kwan00:08:20

More robust compare-and-swap type semantics seems called for here, probably using Datalog query language. Is there a roadmap for this? I'm more than happy to contribute.

refset09:08:57

We don't have plans to extend the transaction primitives currently - but hopefully transaction functions give you all the leverage you need to design your ideal Datalog system on top. Note however that transaction processing is serialised so expensive Datalog queries within transaction functions will reduce the transaction throughput. If you do come up with any particularly neat patterns though it would be great to see/hear about them...and we do accept PRs πŸ™‚

hifumi12302:08:40

I personally use SQUUIDs since they will be ordered by timestamp (and much nicer to search/store in a BST), then on the off chance collisions happen in the same millisecond, have a plan for retrying transactions

hifumi12302:08:53

e.g. when creating entities, if you know a priori these entities are new and do not exist, you can use ::xt/match to check if the eid you generated is currently nil in the document. in effect it lets you do "create if not exists"

Jenny Kwan04:08:14

Thanks! I figured out a retry loop with ::xt/match. And yes, SQUUIDs are way better for my use case.

refset10:08:39

Hi everyone, I'll be running a webinar on the topic of bitemporality next week (more generally, rather than specific to XTDB) - it would be great to see you there, but otherwise please make sure to sign-up anyway and then you can watch/skim the replay at 2x speed ☺️ https://attendee.gotowebinar.com/register/2960607012900067930?source=clojurian-slack

πŸ‘€ 4
refset13:08:54

Hi, quick reminder that this webinar is happening in just over a couple of hours' time if you happen to find yourself available today πŸ™‚ (or you want to catch the replay when that's out!) https://clojurians.slack.com/archives/CG3AM2F7V/p1691057919997759

πŸ™Œ 4
Panel01:08:37

Was it recorded ? I registered but it was at 2am in my tz.

refset09:08:12

Hey @U01SBSXRRH6 yes it was πŸ™‚ the replay will be available soon! We'll be sure to email you once it's ready πŸ™

zeitstein12:08:17

Heads up: 404 on https://docs.xtdb.com/clients/http/openapi/1.24.0. Dropping down a version works.

refset13:08:34

I hit 're-run' on the failed workflow and it went through successfully...and your link looks fixed now :man-shrugging: CircleCI is a real mystery sometimes!

πŸ˜„ 2
seancorfield16:08:23

CircleCI had a temporary outage - it affected http://cljdoc.org jobs as well.