Fork me on GitHub
#xtdb
<
2020-06-19
>
nivekuil06:06:06

congrats on 1.9.0! I don't know if this is new behavior, but I can't seem to figure out how to evict a map id:

rhizome.core> (crux/submit-tx node [[:crux.tx/evict #crux/id  "{:from #uuid \"3e926589-f9ca-470b-b22e-4e64ab8e778d\", :to #uuid \"344a1e62-ae14-44ff-9029-d66903f52b64\"}"]])
#:crux.tx{:tx-id 80, :tx-time #inst "2020-06-19T06:38:16.754-00:00"}
rhizome.core> (crux/q (crux/db node) '{:find [e] :where [[e :from #uuid "3e926589-f9ca-470b-b22e-4e64ab8e778d"]]})
#{[#crux/id "{:from #uuid \"3e926589-f9ca-470b-b22e-4e64ab8e778d\", :to #uuid \"344a1e62-ae14-44ff-9029-d66903f52b64\"}"]}
when I check if the transaction has been commited:
(crux.api/tx-committed? node #:crux.tx{:tx-id 80, :tx-time #inst "2020-06-19T06:38:16.754-00:00"})
Execution error (NodeOutOfSyncException) at crux.node.CruxNode/hasTxCommitted (node.clj:84).
Node hasn't indexed the transaction: requested: Thu Jun 18 23:38:16 PDT 2020, available: Thu Jun 18 22:55:03 PDT 2020
and the node looks like:
(crux/start-node {:crux.node/topology '[crux.standalone/topology crux.http-server/module]
	                :crux.kv/db-dir (str (io/file storage-dir "db"))
                    :crux.http-server/port 8080})

nivekuil07:06:37

Maybe this tx function silently breaks the node?

(crux/submit-tx node
                [[:crux.tx/put
                  {:crux.db/id :delete-where-from
                   :crux.db/fn
                   '(fn [ctx from-id]
                      (let [edges (crux/q
                                   (crux.api/db ctx)
                                   {:find [e] :where [[e :from from-id]]})]
                        (mapv (fn [edge-id] [:crux.tx/delete (first edge-id)]) edges)))}]])
After I try to submit-tx a :delete-where-from the node seems to silently ignore future submit-txs.

jarohen08:06:03

Morning @kevin842 - thanks for getting in touch, we'll see if we can repro this locally

jarohen08:06:22

at first glance - yes, if the node seems to be ignoring future txs after that transaction function it seems like the evict behaviour is a red herring, will focus on the tx-fn for now

jarohen10:06:28

The problem looks to be the compilation error in the transaction function. We don't have the crux alias available when the function's run, so the call to crux/q can't be resolved - needs to be crux.api/q. Hopefully you got an ERROR-level log message in your logs from the crux.tx namespace? That said, a compilation error in a transaction function obviously shouldn't stop transaction consumption - we've released 1.9.1 containing this fix. Thanks for raising the issue!

jarohen10:06:13

If this is a local test database, I'd advise reconstructing it; if it's a non-dev database where this isn't possible, I'd be happy to help out getting it back up and running again?

jarohen11:06:28

Separately, you shouldn't need to include the map ID in a #crux/id reader macro, it should work just supplying the map itself?

nivekuil20:06:19

> Hopefully you got an ERROR-level log message in your logs from the crux.tx namespace? Actually no. This is a separate issue but when I eval crux/start-node in cider I get

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See  for further details.
It's always been that way, I've just ignored it 🙂

nivekuil20:06:30

I'm trying 1.9.1 and I think tx fn still breaks indexing. slack is a little unwieldy so I opened a github issue: https://github.com/juxt/crux/issues/956

jarohen07:06:57

Morning 🙂 thanks for raising the issue. I've replied on there in more detail, but it seems adding await-tx calls fixes the issue - could you check and let me know? Thanks!

ouvasam09:06:46

Hi, Can someone tell me how to use crux http UI ? I did the following but i get nothing on port 3000 as it seems to be the default port

(defn start-node [storage-dir]
  (crux/start-node {:crux.node/topology '[crux.standalone/topology
                                          crux.kv.rocksdb/kv-store]
                    :crux.standalone/event-log-dir "crux-store/eventlog-1"
                    :crux.kv/db-dir (str (io/file "crux-store" "db"))}))

(defonce node (start-node "crux-store"))
(start-http-server node)
I start my server like this lein ring server 5000

jarohen09:06:13

Hey @U0F7M1KA7 - Crux's HTTP server won't get started by lein ring server - that plugin's expecting you to specify a handler in your project.clj, if memory serves? You can start it with a -main function, and then run lein run -m <your-ns>

(defn -main [& args]
  (defonce node (start-node "crux-store")))
For a non-trivial application, you'll probably also want to consider a component-like library to manage the node, rather than using defonce, so that you can stop/start it within the REPL - something like Integrant or Mount (happy to provide more pointers here if you'd like) Re Crux - you can include crux.http-server/module in your :crux.node/topology to have the HTTP server start with your node - have a look at https://opencrux.com/docs#config-start-http-server HTH!

ouvasam11:06:56

Manu thanks @U050V1N74!

🙏 3
jarohen11:06:55

Morning all - we've released 1.9.1 today with a couple of bug fixes, release notes here: https://github.com/juxt/crux/releases/tag/20.06-1.9.1 Thanks to @kevin842 above for raising the above transaction-function issue 🙂

joshkh12:06:33

are there any videos / talks / installation-free demos of crux's built-in UI for browsing data? sounds exciting!

refset12:06:58

great question 🙂 it had a very marginal announcement in the blog because I didn't want to draw too much attention to it ahead of an official launch. There are still some styling and UX issues to resolve before want to promote it more broadly. That said, I would be happy to give you & others a tour if there's interest, and we would certainly appreciate new feedback ahead of the launch. Do you have time today?

joshkh13:06:21

i have a colleague that is interested as well. full disclosure, we are datomic users and have no experience with crux, so we can't provide crux-specific feedback. but we do use an in-house UI browser to explore our own Datomic data and i am curious to see how the pros like Juxt tackled the problem. 🙂

joshkh13:06:01

if all that is okay with you then we are free at 3:30pm (presumably BST?), otherwise we can wait until you gain interest from a bigger crowd

refset13:06:42

@U0GC1C09L Great, it's useful to know the context. Please can you ping me an email and I can set something up: <mailto:[email protected]|[email protected]> (or just send me a calendar invite directly) 3:30 should fine. I am indeed BST also 😎

👍 3
dharrigan14:06:26

Oh missed that!

refset15:06:10

@U11EL3P9U oh, sorry! Was fun though. I'd be happy to re-run for you on Monday/Tuesday, if you like? Send me an email if so 🙂