Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2020-07-09

Channels

#xtdb
<
2020-07-09
>
Vincent Cantin10:07:29

I saw a 9 months old video of a presentation of Crux, and notice the recursive Z pattern used to serialize a 2D time space into a 1D index. It's funny because the same pattern was used by Graphic Cards to store textures in order to minimize the memory cache misses, transforming a square of texture into a linear piece of cache entry.

🆒 3
jarohen16:07:03

today I learned 🙂

Vincent Cantin16:07:41

you will recognize the z pattern of the figures in this paper.

Vincent Cantin10:07:16

I also saw a preview of the GUI in that video, and it's impressive. Crux rocks!

🎸 3
metal 3
🙏 3
ordnungswidrig13:07:14

This was new to me https://neilmadden.blog/2018/08/30/moving-away-from-uuids/ and the author recommends to use secure random 160 bit strings like Xl3S2itovd5CDS7cKSNvml4_ODA. The recent support for string based id is a nice because of this alloned. I wonder if crux should support or advocate for “secure random 160 bit id” in a more prominent matter.

Jorin13:07:31

Guess there are different use cases and different requirements for IDs 🙂 I don't want to judge what Crux is supposed to recommend, but in this context I can recommend this article: https://segment.com/blog/a-brief-history-of-the-uuid/ ksuid can definitely also be an attractive alternative, depending on the use case of course 🙂

đź‘Ť 3
refset13:07:58

Unguessable IDs are certainly interesting from an authorisation perspective (e.g. capability-based security) - I hadn't considered that string IDs open up new options but it makes sense.

jeroenvandijk14:07:18

Thanks for sharing! I’ve seen cases where an UUID is used as a security token, but the validation period was less than a minute and the number of tokens a second would never reach a million. And there might have been other restrictions like IP restrictions. Blindly relying on random token generation because the industry says it’s ultra random is very naive. Also as a general case just relying on a secure random 160 bit will in some (future) cases be naive. So I would not start to advocate and saying this is good or bad in general. People should do their own math. @U054UD60U article should help people remind to do this though!

ordnungswidrig14:07:53

@U0FT7SRLP regarding guessing one shout always take the option into account to temporarily block a resource after a sufficient number of guesses. (This has drawbacks, on it own, though)

jeroenvandijk14:07:42

Yeah maybe rate limiting is a good practise in general. (Who needs to call a remote API more than X amount of times a second)

âž• 3
csm17:07:58

I’m still toying with crux on dynamodb/s3, and have been consistently hitting this error: 

10:01:35.482 [crux-tx-consumer] ERROR crux.tx - Error consuming transactions
java.lang.AssertionError: Assert failed: 0400b444ac06613fc8d63795be9ad0beaf55011936ac
(= (+ c/index-id-size c/id-size Long/BYTES Long/BYTES Long/BYTES) (.capacity k))
	at crux.kv_indexer$decode_entity_PLUS_vt_PLUS_tt_PLUS_tx_id_key_from.invokeStatic(kv_indexer.clj:247)
I’m not sure where to look for what is causing this (tx log? doc store? kv store?)

jarohen17:07:17

Have you upgraded over the 1.8.x -> 1.9.x range with this node, or is this with a clean node?

csm17:07:53

this is a clean node, with one transaction and a single tx event

jarohen17:07:21

Hmm. :thinking_face: it'll be in the kv-store - 04... is one of the bitemporal indices.

jarohen17:07:36

Have you got a repro case you could send through?

csm17:07:08

I’m gonna try starting completely from scratch one more time, the code will all be in github shortly

csm22:07:57

this was my fault; the iterator I’m developing (https://github.com/csm/crux-kv-hitchhiker-tree/blob/trunk/src/crux/kv/hitchhiker_tree.clj) doesn’t work correctly

jarohen08:07:03

Ooh, interesting, thanks 🙂 will keep an eye on that one!