This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-01-25
Channels
- # announcements (6)
- # babashka (53)
- # beginners (199)
- # calva (101)
- # cider (8)
- # clara (17)
- # cljs-dev (1)
- # clojure (86)
- # clojure-australia (1)
- # clojure-dev (33)
- # clojure-europe (29)
- # clojure-france (13)
- # clojure-nl (6)
- # clojure-norway (7)
- # clojure-poland (3)
- # clojure-spec (2)
- # clojure-uk (9)
- # clojurescript (18)
- # community-development (2)
- # cursive (2)
- # datomic (8)
- # duct (43)
- # emacs (9)
- # events (6)
- # figwheel-main (1)
- # fulcro (7)
- # integrant (1)
- # jobs (1)
- # jobs-rus (1)
- # lambdaisland (1)
- # malli (19)
- # nrepl (2)
- # off-topic (31)
- # pathom (35)
- # quil (1)
- # reagent (4)
- # remote-jobs (7)
- # schema (1)
- # shadow-cljs (58)
- # spacemacs (6)
- # startup-in-a-month (12)
- # tools-deps (10)
- # xtdb (40)
@alexmiller I was wondering, why is pipeline
not creating/blocking a thread later in the process, right before running a task. Right now it will create/block-on n
threads even if nothing happens. I think that basically creates higher memory usage for nothing.
it could be like this instead no? https://github.com/mpenet/core.async/commit/e9bb61d (edited to latest version)
that one would cost more in context switches potentially, but at least, when the input chan is empty, 0 thread usage and low memory
I guess that patch could create some more work if the chan always contains values (async/thread call, since it's from a cached thread pool), but it's probably negligible given the usage context
I think with your change, it's no longer bounded by n at all
Need less kids around to be able to play in repl, you put some doubt. But I think since every loop parks on process, every loop never runs more than one task at a time. At least it was the idea
a quick unscientific sprinkling of prn statements over an atom seems to show it's indeed limited to n
tested with that horror in the same ns: https://gist.github.com/mpenet/1c3d9e052543ab1285ddb5ea9fc8285a
@alexmiller should I create a Jira with proper patch & test?
what problem are we trying to solve here?
using fewer threads?
Yes, if you don't have n+ items to consume it will use less threads, potentially 0 if you are waiting on values to be sent to the chan, or when you are dealing with bursty loads.
at the cost of spinning up those threads when needed
you can file something. to really evaluate I would need to dig out the original design stuff for this
I don't think that's going to happen soon
It's possible to emulate this pattern with pipeline-async + thread on the current version. It's a bit ugly but it saves resources when idling
well, I'd lead with the problem
Lastly, since iirc we have a cached thread pool behind thread
the price to "acquire" new threads is low, there's a lot of reuse going on
cached thread pool threads die, so hard to say
that's right