Fork me on GitHub
#xtdb
<
2022-01-28
>
Karol Wójcik07:01:00

Does xtdb has something like squuid in Datomic?

jarohen07:01:02

it doesn't, but if you can generate them XT can store them

Karol Wójcik09:01:27

Hmm. Right, but Squuid guarantees unique identifier between multiple peer instances. How can I ensure the same with XTDB?

jarohen10:01:45

I don't think it does? I thought it was a combination of the timestamp and enough random bytes to be effectively globally unique? happy to be corrected though 🙂

kinleyd12:01:30

Don't know about squuid but why not try ULID (clj-ulid/clj-ulid) - shorter and better guarantees than UUID?

jarohen12:01:55

XT handles UUIDs as 16 bytes - my understanding is that ULIDs are only shorter if you're comparing against UUIDs represented as strings?

kinleyd12:01:30

Yes, as strings from the view point of visual clutter. Also, among other benefits, lexicographically sortable.

Aleed13:01:46

@UJ1339K2B I don’t believe Datomic Cloud has squuid either (been a while). this reference has an example fn that may help: https://github.com/clojure-cookbook/clojure-cookbook/blob/master/01_primitive-data/1-24_uuids.asciidoc

Karol Wójcik13:01:51

I want to guarantee uniqueness for system where there are multiple clients that write to XTDB. As I understand a squuid is unique identifier that is based on time + unique bits + peer bits identifier. https://docs.datomic.com/on-prem/schema/identity.html#squuids Using sequential ID's doesn't guarantee uniqueness in case of many clients writing to XTDB in the same time.

Karol Wójcik13:01:04

This is my intuition however I might be wrong.

kinleyd13:01:22

ULIDs are structured along the same lines you describe for squuids. Plain old UUIDs would pretty much meet your requirements as well, imho.

jarohen13:01:37

@UJ1339K2B: v1 UUIDs do have something akin to a 'peer bits identifier' - it's based on a MAC address on the machine - but I don't think squuids do. might be worth checking on #datomic?

jarohen13:01:14

@U02PJA0AF52: ULIDs seem like a different representation of squuids, which is cool if you want squuids as strings, but wouldn't solve the underlying issue here?

kinleyd13:01:52

@U050V1N74: Yes, I thought ULIDs included MAC addresses but that was incorrect: 48 bits for time and the rest for random bits based on Crockford's Base 32. I'm fairly confident it would solve the underlying issue, though: "1.21e+24 unique ULIDs per millisecond (1,208,925,819,614,629,174,706,176 to be exact)" ;)

fugbix18:01:53

Just in case, since they have adaptive indexes http://squuid%20are%20no%20longer%20required%20in%20Datomic (both Cloud and on-prem).

dvingo19:01:46

right, they mention (https://docs.datomic.com/on-prem/schema/identity.html#squuids) that > Datomic's indexes are optimized to gracefully handle all UUIDs, so Squuids are not strictly necessary. However, you should still prefer Squuids if your ids may ever be indexed in other, non-Datomic systems.

dvingo19:01:29

so I'm curious does XTDB benefit from using sequential UUIDs?

Steven Deobald21:01:52

🎶 Broken record time, but if anyone in the thread wants to summarize this conversation on http://discuss.xtdb.com, someone is bound to thank you months/years from now. 😉

👍 1
refset12:02:16

> so I'm curious does XTDB benefit from using sequential UUIDs? Not to my knowledge, since UUIDs get hashed before being stored in the indexes. I suppose this argument holds though: > you should still prefer Squuids if your ids may ever be indexed in other, non-[XTDB] systems

Steven Deobald15:02:58

I'd be very tempted to qualify that statement as something more like this. 🙂 > "you should still prefer consider Squuids if your ids may ever be will be indexed in other, non-[XTDB] systems" Personally, I would walk a long way to avoid defaulting to any of: sequential, semi-seq, natural, or external ids. If I was only covering for the possibility of putting my data in a database that prefers squuids, I'd have a good long think about the probability of that occurring before I'd actually choose squuids. It would also be a very loud ADR if I did. 😉

👍 1
dvingo19:02:07

thanks for the feedback! very helpful