This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-01-27
Channels
- # announcements (4)
- # asami (6)
- # aws-lambda (1)
- # babashka (38)
- # babashka-sci-dev (20)
- # beginners (87)
- # calva (67)
- # cider (19)
- # clerk (7)
- # clojure (102)
- # clojure-europe (52)
- # clojure-filipino (1)
- # clojure-hungary (4)
- # clojure-nl (1)
- # clojure-norway (6)
- # clojure-sweden (3)
- # clojure-uk (1)
- # cursive (13)
- # data-science (7)
- # datomic (8)
- # deps-new (1)
- # emacs (3)
- # fulcro (16)
- # graphql (3)
- # humbleui (3)
- # kaocha (3)
- # leiningen (3)
- # malli (3)
- # off-topic (14)
- # pathom (34)
- # polylith (4)
- # rdf (12)
- # reitit (3)
- # releases (1)
- # remote-jobs (7)
- # rum (2)
- # sci (22)
- # shadow-cljs (115)
- # tools-deps (26)
- # tree-sitter (29)
Second question of the day! Our index now has a couple hundred resolvers with a few hundred keys. We’re noticing that small changes to queries can have big differences on the performance of the planning step. Are there any principles to keep in mind with respect to improving planning performance?
a big one is to use a persistent cache for the planner, its just adding an atom to the env at ::pcp/plan-cache* key so the plan cache is preserved between requests
since the plan does care about the data values (just their shapes instead), the same query (even with different data) will reuse that cache
Yeah, I added that after I sent over the question and it made a radical difference.
Does the plan cache work independently of the values provided by the query’s ident? If, for example, a :user/username
is provided, will the plan be cached for all possible values of :user/username
?
it supposed to, but I went to check after you said it here, and I see how it isn't working that way
but I think there is a simple fix to it, can you please open an issue?
No problem, I’ll do that now. Can you imagine any side-effects of using a persistent cache for the planner? I can with the resolver cache, but it seems like the planner cache is more or less a free lunch.
thanks for the issue!
the major side effect is memory usage that will increase, but the tradeoff weights a lot more on the gains here
in case the memory usage of the plan cache becomes an issue, then you can switch the basic atom with some more sophisticated caching implementation, an atom is just a type that supports caching, but for pathom what matters is that it implements the cache protocol: https://pathom3.wsscode.com/docs/cache#the-cache-protocol
Gotcha, but with this patch the space taken up in the cache will be considerably less, I imagine.
fix almost done 🙂
Excellent, thank you.
fixed on main
Thanks again. I think Pathom is, hands down, one of the best OSS libraries in the Clojure community, by the way.
humm, Im thinking my solution may have introduced a bug
let me check here
yup, bug confirmed, working on it
I was naive here, because I just changed the cache key, but in the way I did it will always use the same ident value
already have a test exposing it, now time for the fix
current main is working again, but with the ident optimization removed, have to give a bit more thinking to the solution. the issue is that I need to adapt the graph back after, to re-inject the proper ident values there
ok, it should be good now on main
As a recap, it’s all working as you’d like now?
If so, any chance you could cut a new release for Clojars?
(There are two patches I’m hoping to apply.)
yes, it's all running as expected, I plan to cut a release soon, prob today, but max tomorrow
Excellent, thank you!
@wilkerlucio tiny bug at https://github.com/wilkerlucio/pathom3/blob/041daa72b8ced33d2b621ae41371b71ce59bd625/src/main/com/wsscode/pathom3/connect/operation.cljc#L187 - here name
is clojure.core/name
😅 and should have been st. like (::op-name config)
instead.