xtdb 2020-08-08 | Slack Archive

zclj10:08:26

@jarohen two questions regarding a "merge" transaction function. The use case is simply to not reading and updating the complete doc for any update to it. 1. Does this look like a reasonable implementation?

[:crux.tx/put {:crux.db/id :merge-document
               :crux.db/fn '(fn [ctx eid m]
                             (let [db     (crux.api/db ctx)
                                   entity (crux.api/entity db eid)]
                                [[:crux.tx/put (merge entity m)]]))}]

2. When I use the above fn and the merged map contains the same data for any entry in the document, I still get a new valid date but with the same hash. Is this correct behaviour from a Crux point of view? If so, should I fail the transaction function if there is no diff?

{:crux.tx/tx-time #inst "2020-08-08T09:31:28.802-00:00",
 :crux.tx/tx-id 2,
 :crux.db/valid-time #inst "2020-08-08T09:31:28.802-00:00",
 :crux.db/content-hash #crux/id "4d9d84ed3d95eec1b72cd9fa941b0509ae31cad7"}
{:crux.tx/tx-time #inst "2020-08-08T09:37:13.613-00:00",
 :crux.tx/tx-id 3,
 :crux.db/valid-time #inst "2020-08-08T09:37:13.613-00:00",
 :crux.db/content-hash #crux/id "4d9d84ed3d95eec1b72cd9fa941b0509ae31cad7"}

refset11:08:43

#Also sent to the channel

Hi, what you wrote looks good to me, pretty much equivalent to what I wrote in this test: https://github.com/juxt/crux/blob/d988abae614fd86ddf895a1e17d001c48f688d84/crux-test/test/crux/tx_test.clj#L665 Just be conscious that a retroactive (in the past) merge won't cascade through all the subsequent versions. You could certainly add a match operation to avoid the duplicate assertion but I don't think it would provide any improvement on performance or space, however it could avoid the need for skipping over duplicate values when reading through the history

refset11:08:52

What parameters did you pass to the history API, out of interest? It seems right but just want to check :)

zclj13:08:23

(crux.api/entity-history (crux.api/db crux-node) e-id :asc) was the history call used. Considering the history, when I read it as in my example it feels strange that the same hash, i.e. the same data, is valid at two different points in time without a change in between. That seems conceptually wrong, but it might be that I have the wrong expectations on what it means. And just as I'm writing this, I realize that it might be due to a future tx with a valid-time in between the two in my example, Crux can not know that will not happen and need to keep both doc even though nothing has changed for now.

refset11:08:43

replied to a thread:@jarohen two questions regarding a "merge" transaction function. The use case is simply to not reading and updating the complete doc for any update to it. 1. Does this look like a reasonable implementation? [:crux.tx/put {:crux.db/id :merge-document :crux.db/fn '(fn [ctx eid m] (let [db (crux.api/db ctx) entity (crux.api/entity db eid)] [[:crux.tx/put (merge entity m)]]))}] 2. When I use the above fn and the merged map contains the same data for any entry in the document, I still get a new valid date but with the same hash. Is this correct behaviour from a Crux point of view? If so, should I fail the transaction function if there is no diff? {:crux.tx/tx-time #inst "2020-08-08T09:31:28.802-00:00", :crux.tx/tx-id 2, :crux.db/valid-time #inst "2020-08-08T09:31:28.802-00:00", :crux.db/content-hash #crux/id "4d9d84ed3d95eec1b72cd9fa941b0509ae31cad7"} {:crux.tx/tx-time #inst "2020-08-08T09:37:13.613-00:00", :crux.tx/tx-id 3, :crux.db/valid-time #inst "2020-08-08T09:37:13.613-00:00", :crux.db/content-hash #crux/id "4d9d84ed3d95eec1b72cd9fa941b0509ae31cad7"}

Hi, what you wrote looks good to me, pretty much equivalent to what I wrote in this test: https://github.com/juxt/crux/blob/d988abae614fd86ddf895a1e17d001c48f688d84/crux-test/test/crux/tx_test.clj#L665 Just be conscious that a retroactive (in the past) merge won't cascade through all the subsequent versions. You could certainly add a match operation to avoid the duplicate assertion but I don't think it would provide any improvement on performance or space, however it could avoid the need for skipping over duplicate values when reading through the history

pip15:08:15

any good video tutorials for crux? (sorry am a noob) 🙂

refset17:08:06

Hi there! We haven't invested much time in producing video content so far, but we're hoping to change that soon. Did you already see the Crux SQL video? That's probably the closest thing we have to a complete tutorial video at the moment. Also the Nextjournal tutorials are pretty noob-friendly :) Are you planning on using Crux with Clojure?

pip18:08:34

yes trying to figure out if it works for this project but i don’t have much experience with graph databases so have a lot to learn 🙂

pip18:08:49

but thank you! the nextjournal thing looks like a great start for me

pip18:08:08

i saw the clojure/north talk but thats about it

David Pham18:08:02

I wondered if anyone had some experience who could share a bit about Kafka or Postgres or SQLlite as backend: what would be the minimum system/hardware requirements for running crux, with the replication of the data storage for high availability?

2020-08-08

Channels