xtdb 2020-07-02 | Slack Archive

csm03:07:55

hi, I’ve been hacking some more on AWS backends for crux, collected here https://github.com/csm/crux-aws . It’s still all very experimental, but I’m having more success now that when I last looked at this

🚀 15

csm03:07:57

Some relevant individual projects from this: https://github.com/csm/crux-dynamodb https://github.com/csm/crux-ddb-s3 https://github.com/csm/crux-kv-hitchhiker-tree

refset08:07:21

@UFQT3VCF8 this is really interesting stuff! Please do let us know if there's anything we could tweak that might make your work easier 🙂

ordnungswidrig11:07:54

Nice! Did you see my work on dynamodb?

ordnungswidrig11:07:03

https://github.com/ordnungswidrig/crux-dynamodb

csm16:07:03

@U899JBRPF the only thing that would have helped was documentation on the protocols about how they should behave — I had to look at other TxLogs to figure that out. Unless there is a spec for these, and I somehow missed it completely. The protocols are super simple to work with and plug into — that was really awesome!

👍 3

🙏 3

csm16:07:40

@U054UD60U I did recall seeing that before, but it had scrolled off the history. I’m still leaning towards using S3 over DDB as much as possible, since it’s orders-of-magnitude less costly.

csm16:07:22

The KV store is the only part that’s been hard (at least to do serverless). I think using hitchhiker-tree is a good starting point, but I have some issues to iron out with that first.

ordnungswidrig16:07:28

I like the idea of s3 over ddb, bascially the route which datomic took.

ordnungswidrig16:07:27

I was contemplating about kv stores on AWS which could be naturally shared among nodes. Like fully decomposing crux to have a single worker (lambda?) to the TX processing and update the shared kv store or like that.

ordnungswidrig16:07:49

Maybe some layerd kv store which uses memory or local-disk caching backed by the central store.

ordnungswidrig16:07:11

Goal would be a graalvm native-compile fleet of lambda humming happily on demand.

jarohen09:07:05

Morning all 🙂 I've just pushed out an early snapshot of Crux 1.10.0 (`"20.07-1.10.0-(alpha|beta)-SNAPSHOT"`) - fuller release notes will come with the main release, but in the meantime: * SQL module - you can now query Crux with SQL. See https://github.com/juxt/crux/tree/master/crux-sql for how to get started. * Speculative transactions - see what the results of your queries would be if a certain transaction were applied. https://github.com/juxt/crux/blob/master/docs/queries.adoc#speculative-transactions * More types of entity ids - particularly, strings and integers. This one will involve an index version bump - if you haven't already upgraded your production systems to 1.9.0 it might be best at this point to wait for 1.10.0. * Removing polling - you can now plug in non-polling transaction logs (https://github.com/juxt/crux/pull/969), and await-tx no longer polls either (https://github.com/juxt/crux/issues/825). For all the issues/PRs in 1.10.0, take a look at the Github milestone: https://github.com/juxt/crux/milestone/14?closed=1 Feel free to have a play with it - please do let us know what you think, or if you run into any issues!

🎉 24

🚀 3

fatihict10:07:37

Hello 👋, I have a question about a cas failure in crux version 19.07-1.3.0-alpha. I submitted this tx:

[[:crux.tx/put {:some-random-key [{"text" ""}]
                :crux.db/id #uuid"6e603c87-d32b-424c-82ea-46a738931622"}]]

And I'm submitting the following cas:

[[:crux.tx/cas
  (crux.api/entity db #uuid"6e603c87-d32b-424c-82ea-46a738931622")
  {:some-random-key [{"text" "something else"}]
   :crux.db/id uuid}]]

This results in a cas failure, but I don't understand why.

refset10:07:28

Hey! Wow, 19.07 is pretty long in the tooth these days, but I can't see an obvious problem. However just to check, in the second document, does uuid resolve to #uuid"6e603c87-d32b-424c-82ea-46a738931622" also?

refset10:07:11

Out of interest, how come you're wanting to keep working with that version? cas is permanently-deprecated in the latest releases, it's been superceded by match

fatihict10:07:49

uuid does resolve to the same uuid. I forget to change it for this minimal example.

👍 3

fatihict10:07:51

We're working on a couple of features, as soon as those are finished we're planning to upgrade.

refset10:07:38

Ah okay, so you already have something deployed? That's interesting to know.

refset10:07:27

I still can't think of any reason why your cas example would be failing. Can you use a history API to confirm the most recent version is what you think it should be?

fatihict10:07:41

I also work at Avisi 🙂

👌 3

🙂 3

fatihict11:07:32

If my tx was:

[[:crux.tx/put {:some-random-key [{"text" "some value"}]
                :crux.db/id #uuid"6e603c87-d32b-424c-82ea-46a738931622"}]]

instead of an empty string for "some value" the cas succeeds

refset11:07:46

oh interesting, so you think it might be something to do with empty string values? We have changed how the hashing works a few times since 1.3.0, so it wouldn't surprise me if that was a bug at one point

refset11:07:18

If you are able to reproduce that same behaviour on 1.9.1 using cas and/or match then it would definitely be something we need to fix

fatihict11:07:31

Alright, I'll see if I can reproduce it on 1.9.1.

👍 3

euccastro23:07:38

if I transact the exact same document twice does the second attempt mutate the DB in any way?

jarohen23:07:20

Unless you explicitly re-assert the same valid-time, it'll create another entry in the history, yes

👍 6

euccastro23:07:45

thank you!

2020-07-02

Channels