This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-01-05
Channels
- # announcements (1)
- # babashka (61)
- # babashka-sci-dev (1)
- # beginners (54)
- # biff (17)
- # cider (4)
- # circleci (1)
- # clj-commons (39)
- # clj-kondo (26)
- # cljdoc (40)
- # clojure (41)
- # clojure-europe (32)
- # clojure-norway (4)
- # clojure-portugal (1)
- # clojure-uk (2)
- # clojurescript (59)
- # clr (69)
- # conjure (7)
- # cursive (22)
- # data-science (16)
- # datalevin (1)
- # datomic (19)
- # docker (31)
- # funcool (1)
- # honeysql (6)
- # hoplon (1)
- # hyperfiddle (41)
- # introduce-yourself (1)
- # juxt (2)
- # leiningen (5)
- # nbb (14)
- # nextjournal (38)
- # off-topic (47)
- # polylith (2)
- # rdf (5)
- # re-frame (4)
- # reitit (27)
- # releases (6)
- # scittle (10)
- # shadow-cljs (24)
- # sql (11)
- # squint (1)
- # tools-build (33)
- # tree-sitter (4)
- # vim (39)
I use this query to pull out a list of my apps entities (specified by attributes returned by (keys schema)
). It’s quite nice, because I directly know the count and can split it into pages via offset/limit, via a single call.
(d/qseq {:query '[:find (pull ?e [*])
:in $ [?as ...]
:where [?e ?as]]
:args [db (keys schema)]})
But it returns the entities in increasing order (by id). How do I tell the query resolver to start with the highest number? (effectively reversing the result, without loading the whole dataset)So if you want them in a certain order, sort in Clojure then pull, and don’t use qseq+pull
ah wow, thanks. what’s a the better way to bulk pull than this?
(d/q '[:find (pull ?es [*])
:in $ [?es ...]]
db
recently-updated-entities)
it seems quite fast (10x faster than pmap+pull), but feels awkward to use q for itThe speed you see is from avoiding round-trips and blocking; you could recover it by pipelining
Ok, I’m on cloud. I can stick to the q version, which funnily enough needs resorting afterwards again ^^
Yes, on my local machine q
(with consumption) is 30ms and the lines below are 400ms, but maybe that’s not how to use pmap ^^
(->> recently-updated-entities
(pmap #(d/pull db '[*] %))
doall)
pmap is for cpu not io workloads, but on an idle system it’s a quick and dirty way to pipeline even io tasks
(->> (d/q '[:find (pull ?es [*]) ?i
:in $ [[?i ?es]]
db
(into [] (map-indexed vector) recently-updated-entities)
(sort-by peek)
(mapv first))
I've just run into the "Loading database" issue with datomic cloud. Is there any way to prevent the ion instance from receiving requests (lambda, http) until the database has finished loading?
I’m almost certainly misunderstanding the question, because I thought ion deploys were supposed to ensure that DBs were https://docs.datomic.com/cloud/ions/ions-reference.html#deploy.