This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-09-20
Channels
- # 100-days-of-code (2)
- # aleph (53)
- # architecture (2)
- # aws (3)
- # beginners (230)
- # boot (15)
- # calva (3)
- # cider (19)
- # cljs-dev (1)
- # clojure (139)
- # clojure-conj (3)
- # clojure-italy (47)
- # clojure-nl (19)
- # clojure-spec (26)
- # clojure-uk (98)
- # clojurescript (152)
- # clojutre (4)
- # core-async (22)
- # cursive (5)
- # datomic (48)
- # emacs (11)
- # events (1)
- # figwheel-main (219)
- # fulcro (15)
- # instaparse (3)
- # jobs (4)
- # jobs-rus (1)
- # leiningen (30)
- # luminus (8)
- # off-topic (67)
- # onyx (5)
- # pedestal (16)
- # re-frame (1)
- # reagent (4)
- # reitit (31)
- # ring (8)
- # ring-swagger (3)
- # shadow-cljs (115)
- # specter (4)
- # videos (1)
- # vim (20)
- # yada (15)
Hi, I am writing an application consisting of two main parts : a (sort-of) control structure and numerous services, all interacting through channels and implemented via go
blocks. The interaction of services and their life-cycle management is done by the control unit.
And I wanted the thread pool for my control unit's go
s be different from the services. Is there a way I can achieve that ? Or is it even a legitimate concern ?
(The thing is we require that even in case of a failure on part of services, be it thread pool depletion or ..., our control over their life-cycle not to be hindered in any way)
I've seen that also there is : https://stackoverflow.com/questions/18779296/clojure-core-async-any-way-to-control-number-of-threads-in-that-go-thread
Since it wasn't resolved. wanted to check whether there is a better work around - or maybe another way to solve it
if a go ever blocks a thread, that's an error
use parking operations only, clojure.core.async/thread makes a new thread in an expandable pool, and returns a channel you can park on
thanks nosesmith you are right but I wanted to know aside from go blocks blocking threads, in a system heavily dependent on go blocks should I be concerned about separating critical sections' thread pool from others ?
nothing critical should be in a go block
the number of threads used for go blocks is small, and doesn't grow, they are a method of coordination, not a utility for execution / parallelism
But can you elaborate a little more about "nothing critical should be in a go block" and "they are a method of coordination, not a utility for execution / parallelism"
Your comment kinda scared me, cause quite a few of my applications are actually a network of channels with async/go blocks doing almost all of the work in between them
they are for lightweight coordination of state between tasks, anything that uses blocking IO or is CPU intensive should not be in a go block
async/thread is OK for blocking IO or CPU intensive tasks
it returns a channel that a go block can park on
what happens is that when resource-intensive tasks are in go blocks, they can starve the channel operations, since the number of threads for go blocks are limited
At Yummly, we run a local database & dynamoDB during our unit tests, and have the following code hooked into our unit tests to throw exceptions on known blocking operations that we use to make sure we don't end up blocking a core.async thread:
(defn core-async-thread?
"Warn if being run on a main core.async thread."
[f & args]
(let [current-thread (Thread/currentThread)]
(when (re-matches #"async-dispatch-\d+" (.getName current-thread))
(throw (Exception. "Blocking I/O function running on core.async dispatch thread."))))
(apply f args))
(defn apply-core-async-hookes
"Add hooks to various blocking I/O functions to warn if they're being used on
core.async main threads. Safe to be run multiple times as hookes are stored in
a unique set in metadata."
[]
(hooke/add-hook #'korma.db/do-query #'core-async-thread?)
(hooke/add-hook #'korma.core/exec #'core-async-thread?)
(hooke/add-hook #'taoensso.faraday/get-item #'core-async-thread?)
(hooke/add-hook #'taoensso.faraday/put-item #'core-async-thread?)
(hooke/add-hook #'taoensso.faraday/batch-get-item #'core-async-thread?)
(hooke/add-hook #'taoensso.faraday/batch-write-item #'core-async-thread?))
just throwing that there in case it helps anyone, since it's related to the discussion
(require '[robert.hooke :as hooke])
for the hooke/add-hook
function