This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-01-16
Channels
- # ai (3)
- # babashka (3)
- # beginners (252)
- # calva (56)
- # clj-kondo (6)
- # cljfx (7)
- # cljs-dev (2)
- # cljsrn (2)
- # clojure (72)
- # clojure-france (12)
- # clojurescript (13)
- # conjure (60)
- # garden (18)
- # hoplon (16)
- # jobs (1)
- # leiningen (3)
- # off-topic (18)
- # pathom (5)
- # practicalli (1)
- # reagent (4)
- # reitit (1)
- # remote-jobs (1)
- # reveal (3)
- # shadow-cljs (1)
- # spacemacs (7)
- # xtdb (39)
disregard this message
Hi, there 👋 I was following along https://juxt.pro/blog/crux-tutorial-delete Crux tutorial (about delete), but when I try to show the entity history with the :with-docs?
option the evaluation doesn't finish. This part:
(crux/entity-history
(crux/db node #inst "2116-01-01T09")
:kaarlang/clients
:desc
{:with-docs? true})
I'm going to attach a repo case to this message. The crux version I'm using is "20.12-1.13.0-beta"
If it's relevant this is my java version:
$ java --version
openjdk 11.0.9.1 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1+1-post-Debian-1deb10u2)
And my Clojure CLI version:
clojure -Sdescribe
{:version "1.10.1.763"
:config-files ["/usr/local/lib/clojure/deps.edn" "/home/gabo/.clojure/deps.edn" "deps.edn" ]
:config-user "/home/gabo/.clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/usr/local/lib/clojure"
:config-dir "/home/gabo/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:main-aliases ""
:repl-aliases ""}
I believe they just fixed it https://github.com/juxt/crux/pull/1380
Sorry for the noise 😓 I read James's message just a few comments above a run it again with "dev-SNAPSHOT"
and it works
It's not a problem at all, I'm more sorry you hit the issue without better warning - thanks for flagging up that it was causing you a headache! It's good motivation to get the next release out soon, or at least put a note on that section of the tutorial in the meantime 🙂
Hi all, some of my txs aren't getting committed (i'm using custom tx functions but this happens to regular puts, if memory serves) sometimes. What could possibly the cause for a transaction not getting committed?
Hi @U0BR5D7A6 txes should be committed, do you have some example code or a test case, or anything in the logs?
I'm still a bit new, so I don't know if you mean there's a crux-specific log. As for example code, it's a bit hard to nail down. It's nondeterministic as far as I can tell. One moment txes are working, the next moment they're not. In general, restarting a server is enough to make it work, but there are times when it just fixes itself after some time.
In all instances that this has happened though, queries are fine. It's just transactions that go wonky
By default txes are processed asynchronously, you need to call await-tx if you want to ensure a tx has been processed.
Alright, I'll try that! thanks!
What i've been doing up until now was a combination of
submit-tx
and sync
. Isn't this supposed to be roughly the same as await-tx
though?
sync will ensure a node is caught up at the time you call it, there's still a race condition. Await-tx for a specific tx is what you need.
Hope that helps @U0BR5D7A6
Hmm, so i'm still getting the issue even with await-tx, and it's also nondeterministic. @_@
However, once i'm in a situation where a transaction doesn't get committed, no transaction will be committed
@U0BR5D7A6 A sample test case that reproduces the issues (even inconsistently) and the version number of the Crux you're using would really help. This sounds pretty peculiar, so it might take more than a description to nail it down.
Alright, now that I know that there's no known reason that could easily explain this behavior, then I'll invest in a day or two of narrowing it down, then file an issue. 🙂
Hi 🙂 if you can share the Crux version and the config options you're passing to start-node
in the meantime that may well help to triangulate the issue
Hallo! I'm on the dev-SNAPSHOT
, and the config options are:
{:crux.jdbc/connection-pool {:dialect {:crux/module 'crux.jdbc.psql/->dialect}
:db-spec {:jdbcUrl
(:jdbc-database-url config)}
#_#_:pool-opts {}}
:crux/tx-log {:crux/module 'crux.jdbc/->tx-log
:connection-pool :crux.jdbc/connection-pool}
:crux/document-store {:crux/module 'crux.jdbc/->document-store
:connection-pool :crux.jdbc/connection-pool}}
@U0BR5D7A6 thanks, do you have the code used for submitting txes?
I figured out the problem, and while I can't conclusively say this explains all the behaviors, this should pretty much be it.
• A race condition was causing a custom-defined function to be sometimes present or absent.
• (crux/submit-tx node [[:crux.tx/fn :any-custom-fn-that-doesnt-exist]])
will fail silently
• The tx will of course not be committed.
If the silent failure is a heretofore unknown issue, then i'll file an issue in the github with a repro repo.
Details
• I'm deploying on heroku, where I wanted to make sure I could save as much on database rows as possible.
• I don't need historical versions of my custom functions, so I was evicting them before putting them in.
• evict was taking almost 30 seconds even if it should only realistically be deleting one record per custom function
• I was using await-tx elsewhere, but had neglected to use it in this situation. await-tx on the evict submission before the puts on the custom functions seems to have prevented the issue altogether.
It's a workaround for replace, rather than append. I've placed the puts of the custom functions in my application's initialization code. So if I don't put any cleanup code, the database will constantly be accumulating the history of the custom function.
And to my knowledge, there's no way to mark a specific entity as "do not store history"
How about you do it at application bootstrap and check for the function being equal to itself before putting in a new one?
Do you mean the function not being equal to itself? By my understanding, match cannot do this, and so a custom function must be used, and we’re back to square one. :rolling_on_the_floor_laughing:
Alternately, you don't need to be atomic, do you? Just crux/entity your key and (=) the result
My understanding of match is, it will make a transaction proceed IF it matches a certain value. I cannot make a transaction proceed if it doesn't match a certain value. In a way, it's a (when (= db-value match-value))
but what I need is (when (not= db-value match-value))
.
Yes, using crux/entity would serve the not=
functionality that I need, but the broader problem is that I don't need historical version of a custom function at all. And even with the not=
, over time this would accumulate history that I don't need, presumably as I change implementations.
Having said that, is there any sort of ideological reason one shouldn't use evict for the purpose of replacing a document?
I would think that evict, being a tx, still takes up space on the tx log, so you're better off looking before you leap if you want to save space. I think pruning history would better be done as some kind of periodic compaction job by some external process
Who cares if there is a history maintained of the past versions of the function? You are right match doesn't negative-match, but you can call entity and confirm the not= yourself.
I care when my project is hosted on heroku, and I have to pay extra when my database rows go over budget 😛
Yup! Which is why I went with evicting the custom functions and putting them in at application bootstrap 😆