This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-09-20
Channels
- # 100-days-of-code (2)
- # aleph (53)
- # architecture (2)
- # aws (3)
- # beginners (230)
- # boot (15)
- # calva (3)
- # cider (19)
- # cljs-dev (1)
- # clojure (139)
- # clojure-conj (3)
- # clojure-italy (47)
- # clojure-nl (19)
- # clojure-spec (26)
- # clojure-uk (98)
- # clojurescript (152)
- # clojutre (4)
- # core-async (22)
- # cursive (5)
- # datomic (48)
- # emacs (11)
- # events (1)
- # figwheel-main (219)
- # fulcro (15)
- # instaparse (3)
- # jobs (4)
- # jobs-rus (1)
- # leiningen (30)
- # luminus (8)
- # off-topic (67)
- # onyx (5)
- # pedestal (16)
- # re-frame (1)
- # reagent (4)
- # reitit (31)
- # ring (8)
- # ring-swagger (3)
- # shadow-cljs (115)
- # specter (4)
- # videos (1)
- # vim (20)
- # yada (15)
We ran into a surprising corner-case where if a deferred chain has a manifold from alia (from alia.manifold/execute) in the chain, the subsequent fns in the chain will be executed on the cassandra java-driver pool, not the manifold pool
but it's a bit unclear right now how we could keep the benefits of d/chain while avoiding this behavior (which can quickly exhaust the small I/O callback pools used by cassandra-driver)
if I recall you can pass an :executor option to alia.manifold/execute to have the callback run in another pool
I can track that down to (d/on-realized (alia.manifold/execute ...) #(warn %) #(warn %)) as showing the callback running on the cassandra-driver netty threadpool
(the result is a ListenableFuture which extends Future which manifold considers as a valid deferred)
I'd have to check how things work internally in manifold, I am not too familiar with it.
huh, i'm surprised i've never run in to that problem
what made you notice it @pyr?
in alia we have the callback (guava future) run in execute currentThread, unless specified otherwise.
@mccraigmccraig exhausting cassandra-driver's pool
does the cassandra-driver pool throw when it's exhausted ?
Do you think alia should by default expose/use a callback threadpool to avoid these kind of odd cases?
If I recall I followed what java-driver does, not creating an additional pool for this and leaving it to the user
at some point the result is picked up as homomorphic to a deferred (makes sense) but the listenablefuture's callback pool is then used as the executor (doesn't yet make sense to me :-))
I get highlights/notifications on "alia" mentions, looking fwd to understanding what's up
The corresponding deferred thus has no executor, which means that the callbacks will be executed on the calling thread, hence in this case on cassandra-driver's threadpool
oh right, so in theory just passing the :executor option value to deferred should do it, then the alia/execute calling thread would be the default and the user can overwrite that with that :executor option if needed
right: https://github.com/ztellman/manifold/blob/master/src/manifold/deferred.clj#L562