This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-04-11
Channels
- # aws (6)
- # beginners (167)
- # cider (41)
- # cljs-dev (6)
- # cljsrn (3)
- # clojure (399)
- # clojure-dusseldorf (1)
- # clojure-nl (2)
- # clojure-spec (3)
- # clojure-uk (47)
- # clojurescript (16)
- # core-async (8)
- # cursive (56)
- # datomic (14)
- # devcards (1)
- # docs (2)
- # duct (2)
- # editors (3)
- # emacs (3)
- # fulcro (178)
- # graphql (10)
- # off-topic (107)
- # onyx (7)
- # pedestal (21)
- # planck (13)
- # re-frame (58)
- # reagent (76)
- # ring-swagger (3)
- # shadow-cljs (85)
- # slack-help (2)
- # sql (1)
- # tools-deps (11)
- # uncomplicate (5)
- # vim (24)
- # yada (4)
What’s the state of the art for tuning a pipeline (in the generic sense)? For example, with two processes are parallelized using async/pipeline
and friends, what tools/techniques do folks use to determine how big the channel buffer should be, the number of threads to parallelize for each, etc? I’m curious about making sure that threads from either pipeline
call aren’t just sitting around.
Put another way, how can I get visibility into the “backpressure” that my system may be exerting?
if you are using pipeline (not pipeline blocking) there aren't any threads sitting around
pipeline runs things in go loops, so they reused the same fixed size threadpool that runs go loops, and that threadpool is limited to 8 threads by default
I wrote a thing that includes some examination of the flavors of pipelines https://ce2144dc-f7c9-4f54-8fb6-7321a4c318db.s3.amazonaws.com/buffer.html the naive modeling equations are not correct (the data in the tables don't match them)