Fork me on GitHub
#xtdb
<
2021-08-17
>
seancorfield00:08:42

@richiardiandrea I would expect to call .stop on a running Jetty server, not .close.

richiardiandrea02:08:32

Oh I see yeah, probably I just don't receive back the Jetty server instance there

tatut05:08:00

coming from datomic, it takes a little getting used to thinking about documents instead of entities... in datomic you can add stuff to existing entities without caring about other data it already has, documents afaict are replaced whole

tatut05:08:03

it seems there isn't a builtin "patch" (something like (merge-with merge) ) tx function... what's the common pattern? make a tx function for that?

Tuomas05:08:25

Yeah tx function.

✔️ 3
Tuomas05:08:28

If you q, merge and submit, then you risk overwriting unindexed transactions

tatut05:08:14

thanks, that makes sense... I was just surprised that there wasn't a builtin for that

4
tatut05:08:28

perhaps the way you merge documents is app dependent and not generic enough to be a builtin

✔️ 3
Tuomas05:08:01

Yeah I'd argue otherwise, but on the other hand only having the most basic tx primitives pushes you to use tx functions in a really early stage and opens you up to the world of possibilities

Tuomas05:08:24

Oh I forgot. You can q, merge and submit with :crux.tx/match . That approach requires factoring in the possibility of match failing (retry/give up). Thinking from this perspective patching really is kinda app dependent. In a tx function you would add to yet unindexed changes and with match your would only update if there are no new changes on their way. Giving a :crux.tx/patch would guide everyone to choose the former although matching seems valid in some cases too

tatut06:08:39

so with match you have CAS and you can make your own tx fn for a more fine grained patching

refset09:08:39

I think there are two reasons we haven't added patch first-class: • minimizing API surface area wherever we can • adding a new built-in operation to simply patch against "now" is ~trivial but arguably incomplete - temporally-aware patch semantics (i.e. inserting into the past or future) is non-trivial to reason about so it would require a fair bit of work to ship something that we'd consider "complete"

🙏 6
richiardiandrea15:08:16

I have to say that is something I was also longing for. Patching is not that trivial (or I am doing something very wrong here) if you also have keys that are not "modifiable"...not that difficult either but now you have to think about it more than you probably should

👌 3
kevinmershon16:08:32

We're looking for differential updates as well. Ideally, with the entity-history also showing a field with the deltas rather than only the state of the whole document

3
refset17:08:45

Thank you everyone for all the feedback so far - please keep it coming! > Ideally, with the entity-history also showing a field with the deltas rather than only the state of the whole document Would you expect this to work by storing the explicit deltas somewhere (e.g. as separate documents or nested within some tx-meta doc), or by doing some post-facto diffing on-the-fly (using some approach like recommended in this post which I've just skimmed very briefly https://juji.io/blog/comparing-clojure-diff-libraries/)?

kevinmershon17:08:06

@U899JBRPF So, one of the things that would be valuable to us is the ability to basically perform a "git blame" on a field, e.g. query for "when did this field on this record change" and get back a transaction id, the deltas within that transaction that happened on that record, and the state of the record prior to the change

👍 3
kevinmershon17:08:34

So presumably that would require the delta to be stored rather than calculated.

richiardiandrea18:08:05

As a comment, the pattern of storing a tx-meta doc would definitely solve my problem of "system computed" (like updated-by) vs "modifiable" keys when updating an entity probably

👍 3
Tuomas05:08:23

Crux supports a handful of eql parameters https://opencrux.com/reference/1.18.0/queries.html#_attribute_parameters Is it possible to write your own parameters? I'm thinking of :offset at the moment but I see a lot of value in the ability to add bespoke parameters also

refset17:08:36

Hi @UH9091BLY I can't see any way for users to extend these parameters currently (without forking crux.pull https://github.com/juxt/crux/blob/master/crux-core/src/crux/pull.clj) but it's something we could consider opening up somehow. Can you describe an example of what you would expect :offset to do?

Tuomas17:08:09

Id expect it to work the same way as :offset works in q. Limit 1 gives the first child, limit 1 offset 1 gives the second. Limit, offset and maybe order-by seem like something you could have out of the box, but being able to implement something like :todo/checked? could be powerful too

refset11:08:29

I see, interesting :thinking_face: Well, since the pull evaluation works against documents and not indexes, there's probably not a huge difference, performance-wise, between doing limit/offset natively vs as a post-processing walk. Do you already have a post-processing workaround in place?

Tuomas09:08:21

I have gql in front of crux and atm I'm planning on resolving joins with q's (1q / doc and 1q / join). Resolving refs to others docs and docs that reference a given doc with a given attribute is pretty simple. In theory it should work fine for shallow gql queries with small results

Tuomas09:08:27

For deep queries and large results I might need to look into how combine everything into a single q

refset10:08:24

ah I see, we have certainly had similar thoughts about GraphQL over the last couple of years. A single query would definitely be preferable unless the query was so large & slow that it made sense to cache certain parts of it and therefore decompose it into separate queries again 🙂

refset10:08:12

feel free to open a discussion/issue on Github with your request, I can't make any promises that we can do anything about it, but perhaps it will inspire some collaboration and new ideas