This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-28
Channels
- # babashka (167)
- # beginners (91)
- # calva (24)
- # chlorine-clover (5)
- # cider (14)
- # clj-kondo (15)
- # cljdoc (20)
- # clojure (122)
- # clojure-czech (1)
- # clojure-europe (31)
- # clojure-france (2)
- # clojure-nl (5)
- # clojure-spec (8)
- # clojure-uk (7)
- # clojurescript (29)
- # conjure (2)
- # cursive (4)
- # data-science (4)
- # datomic (13)
- # figwheel-main (13)
- # fulcro (21)
- # lambdaisland (4)
- # meander (10)
- # observability (7)
- # off-topic (15)
- # overtone (4)
- # pathom (5)
- # pedestal (6)
- # re-frame (9)
- # reitit (13)
- # remote-jobs (2)
- # ring (1)
- # rum (5)
- # shadow-cljs (24)
- # spacemacs (19)
- # test-check (18)
- # tools-deps (82)
- # tree-sitter (1)
- # xtdb (35)
Hello, what would be a recommended way to check if my node is still connected to the storage? I have a node backed by Postgres and wanna write a health check in my service which alarms if the connection is broken/postgres is down etc
as far as I could tell, the "Monitoring" part of the crux docs only talks about gathering metrics, not any healthchecks, so I ended up writing a function that makes a specific query and if there is zero or more results, return true and if there is any exceptions, return false. I also use that to measure latency over time
Right, but does doing a query guarantee that i goes all the way to the storage? I was thinking some of them maybe could be answered from the local node index.
Hm, good question, don't think it guarantees that. Maybe checkout the output of (crux/status)
in different failure scenarios, if you could use that?
I did try exactly that but the status seems to return fine even when the postgres instance is down
I'm almost inclined to do a raw JDBC query SELECT 1 + 1;
and see if that bombs. Was looking for something better hopefully?
Hey @U7ERLH6JX - I agree some clearer health check for crux-jdbc would be useful. I will have a chat to the team. In the meantime I know open-tx-log
is guaranteed to access the underling JDBC-powered tx-log
awesome! thanks a lot! i can use that for now. also if you could point me in some right directions I'm happy to raise my first PR to Crux! 😄
More beginner questions, when I'm using :crux.tx/delete
it seems my queries still return the thing I'm deleting but if I use :crux.tx/evict
future queries won't show it. Do I need to adjust my queries to take deletions into account?
That doesn't right at all 🙂 delete should stop later queries seeing the deleted entity. Are you definitely querying against a db value that is take at a later vt/tt time when the delete will have taken effect? If you can show an example that would help Evict is probably not something you want to use unless you really need to
Hurrah, then I'm not completely lost, probably just a little lost! Here is a sample: https://gist.github.com/victorb/27f74557f8f452880b2c95126697f9f1
here is how I start the crux node, it's basically all rocksdb for now:
(def crux-opts
{:my-rocksdb {:crux/module 'crux.rocksdb/->kv-store
:db-dir (io/file db-path)}
:crux/tx-log {:kv-store :my-rocksdb}
:crux/document-store {:kv-store :my-rocksdb}
:crux/index-store {:kv-store :my-rocksdb}})
Hm, so, stumbled upon
Nope. But definitely something wrong with my query rather than the deletecrux/await-tx
(via https://juxt.pro/blog/crux-tutorial-await) and if I wrap the delete transaction in await-tx, things seems to work as I expect...
A smaller example. I don't quite get why the delete works as I expect when there is just the :crux.db/id + :name attribute but if I add :age it shows up in the query even after running the delete
(crux/submit-tx (crux-node) [[:crux.tx/put {:crux.db/id :my-id
:name "hello"
:age 17}]])
(crux/q
(crux/db (crux-node))
{:find '[n]
:where '[[n :name name]
[n :age age]]
:args [{'name "hello"
'age 17}]})
(crux/submit-tx (crux-node) [[:crux.tx/delete :my-id]]))
after running out of options to try, I tried replacing :args
with :in
and now the query doesn't show the deleted result...
What's the difference between :args and :in? Can't find anything in the docs
Hi again, I'm still trying to sanity check your example 🙂 I wrote this for random-hex - presumably you had something similar:
(defn random-hex [n] (doall (apply str (doall (repeatedly n (partial rand-nth "0123456789ABCDEF"))))))
But the key difference between :in and :args is that :in isn't officially supported yet 😉@U899JBRPF take a look at the smaller example posted later, think that showcases (my misunderstanding of something) better. Hah, got it. Somehow it shows "old" data when using :args
but shows no data (correct) when using :in
:in is technically superior but we couldn't implement previously until we had implemented destructuring (we which have now done in 1.12)
Smaller example with :args
commented out. If you run it with :args
you'll see the data still gets returned from the query, but if you run it with :in
(identical structure as with :args
) after the delete, it's no longer visible...
(def crux-node (crux/start-node {}))
(crux/submit-tx crux-node [[:crux.tx/put {:crux.db/id :my-id
:name "hello"
:age 17}]])
(crux/q
(crux/db crux-node)
{:find '[n user-name age]
:where '[[n :name user-name]
[n :age age]]
;; :args [{'user-name "hello"
;; 'age 17
:in [{'user-name "hello"
'age 17}]})
(crux/submit-tx crux-node [[:crux.tx/delete :my-id]]))
hmm, that's not how :in
works and gives an error for me. Which version is this? Definitely 1.12? You can see via status
Thanks for figuring out the smaller example. I have managed to reproduce this issue now. Looks a lot like a bug to me! I'll keep you posted
Hoho, I'm (more or less) glad to hear that it wasn't me doing something absolutely ridiculous. Thanks for looking into it!
Two final questions: is this a regression or newly found issue? Feels like a pretty basic use case that it'd be weird that it hasn't been found before, or is there something else I should use instead of :args
that is more idiomatic?
We'll figure out if there's been a regression in due course, but it looks like a fairly niche corner-case to me as I think it seems to only trigger when there's an exactly matching arg var for all values that actually belong to the entity. :args
is the right way to do things to benefit from query compilation caches, however :in
will likely be available in the next release (I misunderstood the status of it when I referred to :in
earlier)
Thanks a lot for explaining! Seems it's been fixed in master now (https://github.com/juxt/crux/pull/1130) cheers for fixing it so quickly! 👏
If anyone is interested in a java only KV store I just released a new version for crux-xodus: https://github.com/avisi-apps/crux-xodus/releases/tag/1.0.0 This one is compatible with the new 1.12 release from Crux
after running out of options to try, I tried replacing :args
with :in
and now the query doesn't show the deleted result...
What's the difference between :args and :in? Can't find anything in the docs