Fork me on GitHub
#xtdb
<
2020-07-23
>
sveri18:07:24

Hi, so I just wanted to try crux and wonder what I am doing wrong. I have these few lines:

(defn start-standalone-node ^crux.api.ICruxAPI []
  (crux/start-node {:crux.node/topology '[crux.standalone/topology]
                    :crux.kv/db-dir     (str (io/file "test-storage" "db"))}))

(let [node (start-standalone-node)]
    (crux/submit-tx node [[:crux.tx/put {:crux.db/id :ivan, :age 40}]])
    (println (crux/entity (crux/db node) :ivan)))
and the entity call returns nil. I also see that: https://nextjournal.com/crux-tutorial/start the entity call returns nil in the browser. So I wonder if something changed in the API maybe?

sveri18:07:59

I am using the RELEASE version, which resolves to: 20.07-1.9.2-beta

ordnungswidrig18:07:30

@sveri the transaction maybe wasn’t processed yet. Use await-tx to wait for the changes to show up in the indexes:

(crux/await-tx node (crux/submit-tx node [[:crux.tx/put {,,,}]]))

ordnungswidrig19:07:17

There’s chapter in the tutorial about it https://juxt.pro/blog/crux-tutorial-await

ordnungswidrig19:07:50

Interestingly in the tutorial the same “error” shows up and the result is nil while the entity is expected according to the prose. I think crux might have changed behavior regarding when a tx is visible (/cc @taylor.jeremydavid)

😅 3
sveri19:07:23

@ordnungswidrig Indeed, using await-tx it works as expected, thank you. Well, as a newcomer, a hint would be nice in the Get Started section.

👍 3
refset19:07:55

Oops, yeah that tutorial needs an await-tx - sorry! It's possible Nextjournal changed their execution model so it's synchronous now but either way I'll see that it's fixed

3
ordnungswidrig19:07:21

@taylor.jeremydavid what’s the reason behind being “forcing” await-tx be default?

refset19:07:22

That's a very good question and in fact you would find a submit+await-tx helper function in most of the Crux-related code we write. It's primarily because like to be very cautious about our long-term API support. I don't know what the tipping point might be for bundling such a function into the core...but we are certainly overdue to give it some consideration! Feel free to open an issue if you fancy arguing the case and collecting some +1s, or I will raise it with the team anyway over the next couple of weeks 🙂

ordnungswidrig19:07:19

Now that I think about it, it’s more like trap for the young players. Similar to lazy sequences in clojure.

jarohen20:07:34

It's mostly to improve ingest speed - if you were to have await-tx be default each individual tx in a large ingest job would have to wait to be indexed by the current node before submitting the next one. That said, we can certainly make it both more ergonomic (we've talked about a few options but, as @taylor.jeremydavid says, given how core this API is it's not something we want to change without due consideration) and clearer in our docs 🙂

sveri20:07:33

So, from my experience today, a clear pointer in the getting started section would have been enough for me personally 🙂

ordnungswidrig06:07:16

@U050V1N74 yeah it's like autocommit in RDS which having it is a good default but if might kill your for batch loads or when you need atomicity around a batch of statements. Crux just went with the opposite default, which is a good decision IMHO.

ordnungswidrig06:07:27

I wonder if submir-tx could return a reference which when derefed would block until the tx is indexed? (-> (submit-tx node [,,,]) :await-tx) deref) that would allow to use primitives to wait for a collection of reference to be realized or a certain amount etc. Similar to promises or channels. But with composition. (And something with monads).

ordnungswidrig06:07:41

Use case would be e.g. batch processing where for each step you might to want to wait for a couple of tx before proceeding.

jarohen10:07:12

I do quite like the idea of deref as a syntax here - thanks 🙂

dominicm21:07:14

Datomic uses deref for this too

dominicm21:07:21

It's a natural fit

ordnungswidrig19:07:43

It’s really surprising when comparing with other databases where tx per call is the default.