This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-02-08
Channels
- # aleph (7)
- # announcements (12)
- # babashka (19)
- # beginners (4)
- # calva (29)
- # cider (20)
- # clj-kondo (20)
- # clojure (66)
- # clojure-austin (4)
- # clojure-europe (11)
- # clojure-nl (1)
- # clojure-norway (42)
- # clojure-uk (4)
- # clojuredesign-podcast (9)
- # conjure (1)
- # cursive (5)
- # datomic (42)
- # etaoin (4)
- # events (10)
- # garden (8)
- # graphql (1)
- # holy-lambda (7)
- # honeysql (3)
- # hyperfiddle (5)
- # missionary (11)
- # music (1)
- # off-topic (12)
- # practicalli (2)
- # re-frame (2)
- # reitit (6)
- # releases (2)
- # vim (2)
- # web-security (1)
- # xtdb (3)
Valcache seems to take a very long time to complete its startup - on my local, it’s around 3 minutes. I’m wondering if it’s a quirk of my local setup, or a known issue?
2024-02-08 09:29:50 INFO d.valcache-direct {:event :valcache-direct/start, :root "/Users/dazld/valcache/", :pid 12712, :tid 21}
2024-02-08 09:29:50 INFO d.valcache {:event :valcache/direct-init, :datomic.valcache/path "/Users/dazld/valcache/", :datomic.valcache/eviction-threshold-mb 4000, :pid 12712, :tid 21}
2024-02-08 09:29:51 INFO d.async {:event :daemon/thread-started, :name "valcache-eviction-loop-1", :pid 12712, :tid 54}
2024-02-08 09:29:51 INFO d.valcache {:event "ValcacheEvictPlan", :path "/Users/dazld/valcache/", :threshold-mb 4000, :interval-secs 10, :file-window 10000, :pid 12712, :tid 54}
2024-02-08 09:32:36 INFO d.c.datomic :connected
MacOS, M1 Pro with 16GB.We use a pattern for filtering queries (built in code by user input). To make this tractable we have a pattern of a rule per filter, e.g.
[(filter-entity-by-rel1 [?test-value ?entity-id])
[(!= :none ?test-value)]
[?entity-id :entity/rel1 ?test-value]]
[(filter-entity-by-rel1 [?test-value ?entity-id])
[(= :none ?test-value)]
(not [?entity-id :entity/rel1])]
In this pattern ?test-value
may be either an entity id or the :none
sentinel (which is not an ident in the database).
I am finding cases where the :none
guard clause seems to be moved around unpredictably so that evaluates after the pattern clause, throwing an exception. This filter in a query by itself runs fine, but some combination with other clauses in the query can (reliably, but unpredictably) cause it to throw java.lang.IllegalArgumentException: Cannot resolve key: :none
, which looks a lot like it evaluated [?entity-id :entity/rel1 ?test-value]
first. Is there a reliable way to implement this pattern short of using a negative integer as a none
sentinel?Hate to ask, but any chance you have a minimal repro ? Would go a long way in speeding up the analysis.
[(filter-entity-by-rel1 [?test-value ?entity-id])
[(!= :none ?test-value)]
[(identity ?test-value) ?tv2]
[?entity-id :entity/rel1 ?tv2]]
(d/query {:query '[:find (pull ?e [:entity/id])
:in $ % [?filter ...] ;; the only change
:where
[?e :entity/id]
(filter-entity-by-rel1 ?filter ?e)]
:args [(d/db c)
'[[(filter-entity-by-rel1 ?test-value ?entity-id)
[(!= :none ?test-value)]
[?entity-id :entity/rel1 ?test-value]]
[(filter-entity-by-rel1 ?test-value ?entity-id)
[(= :none ?test-value)]
(not [?entity-id :entity/rel1])]]
[:none]] ;; now wrapped in a vector
})
Thanks @U09R86PA4
Any advice here? Are guards something that is supposed to work, or is reordering as preds a fact of life that isn’t going to change? If the latter, what do you suggest for patterns like this that are entity + sentinel? Wrap in a tuple to tag the variant?
I haven’t had the bandwidth to look into this yet, but the exception appears to be trying to resolve your :none keyword as an entity/ident
Yes, correct, I assume that is because it became a :pred and evaluates after the pattern clause now
I use guard predicates like this a lot in rules with multiple implementations, and it had been reliable, but now I’m wondering if it was meant to be reliable and I should do something else.
Hey Francis,
A few things seem to be going on here. For one, the the query engine is attempting to resolve ?test-value
in the clause [?entity-id :entity/rel1 ?test-value]
as if it's an ident because :entity/rel1
is a :db.type/ref
. The next question I would ask is: why are we reaching that clause if [(!= :none ?test-value)]
returns false?
This actually appears to be related to how datalog rules are processed - a clause that returns an empty result set doesn't cause the rule to short-circuit. You can see this behavior with query-stats.
Notice how :rows-out
is 0
for all clauses inside of this rule, yet all of the clauses are still present and have been processed. I imagine this normally isn't noticeable when your clauses don't result in an error, but in the case you presented, the failure to resolve :none
as an ident is making this behavior apparent. Does that make sense?
Sure, but why does it sometimes work and sometimes not? I made a trivial change (adding destructuring to the input) and it behaves as expected.
Really I’m looking for guidance on how one ought to structure rules that have this sentinel-or-entity-id pattern
I don’t know if this is something I shouldn’t expect to work reliably ever (i.e. a binding mentioned in a pattern clause can only safely be an entity ref), or if this is a bug.
The behavior of a collection binding and how that doesn't manifest the same error is very interesting. I'll have to get back to you with more, but wanted to respond so as to not leave you hanging in the dark 🙂
For datomic pro (with transactor deployed with app, connecting to postgresqldb) I'm getting error AMQ219007
when I'm starting second app (with transactor) trying to connect to the same datomic-db (with the same name and that transactor connects to the same postgresqldb)
How can I solve this, to be able to have two transactors at the same time? Is there some configuration I'm missing?
transactor host
and alt-host
values from the transactor.properties file are written into storage. They need to be different values for each transactor
only one transactor will be active at a time; the second will be a hot spare (https://docs.datomic.com/pro/operation/ha.html)
peers find and connect to transactors by reading the same host and alt-host values from storage
so, everything depends on host and alt-host values: make sure they are different for each transactor, and make sure peers and transactors can all resolve those values, or at least one of them, to a transactor instance.
If you are trying to isolate peers to different sets of transactors, know that that is not possible
@U09R86PA4 Thank you very much! It is all clear now
I just want to double check that it's not possible to pass in a separate database, other than the implicit db, to a pull
inside the :find specification. Judging by the grammar (https://docs.datomic.com/pro/query/query.html#grammar) this is not a possibility.
In my case, the where clause uses a filtered-db and I'd like the pull expression to use an unfiltered db. No biggie if this doesn't work of course -- I'll just just pull-many after the fact
Yeah! Thanks Favila -- right after I asked I figured I might as well give it a shot, and like you said, no errors are coming out. Gonna see if it works as expected
I was using this syntax for quite a while before I realized it wasn’t supported--so I discovered it by accident