This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-05-22
Channels
- # aws (5)
- # bangalore-clj (3)
- # beginners (84)
- # boot (31)
- # carry (1)
- # cider (6)
- # cljsrn (7)
- # clojure (188)
- # clojure-conj (1)
- # clojure-dev (3)
- # clojure-dusseldorf (8)
- # clojure-greece (10)
- # clojure-italy (7)
- # clojure-nl (3)
- # clojure-russia (45)
- # clojure-spec (5)
- # clojure-uk (64)
- # clojurescript (83)
- # clr (9)
- # core-async (45)
- # cursive (55)
- # datomic (8)
- # devops (1)
- # emacs (7)
- # graphql (1)
- # hoplon (3)
- # jobs (1)
- # jobs-discuss (37)
- # lein-figwheel (3)
- # leiningen (4)
- # luminus (6)
- # off-topic (5)
- # om (2)
- # onyx (33)
- # pedestal (13)
- # re-frame (9)
- # reagent (28)
- # rum (15)
- # specter (4)
- # unrepl (38)
- # vim (3)
Hello all, is that possible to use core.async as a queue? I´d like to start 10 threads and do some action after the 10th thread finishing.
how to do that? I found this in clojure docs
(doseq [n (range 10)
:let [i (-> n
inc
range
rand-nth)]]
(go
(<! (timeout (* i 1000)))
(println n)))
Is that something like this? replace i to thread result and <! is for waiting all threads terminate?you might want to look at pipeline https://clojuredocs.org/clojure.core.async/pipeline
@fabrao IMO using core.async simply for creating/joining threads is overkill. The following code starts 10 threads, joins them (“join” means “wait for”), and then moves on — there are many ways to do this, but this is probably the easiest:
(let [tp (java.util.concurrent.Executors/newCachedThreadPool)
your-fn #(println "Hooray!")]
(.invokeAll tp (repeat 10 your-fn))
(println "Everyone's done!"))
If you’re already using core.async, then awesome — otherwise, you are going to be using it for something that’s already built into the core language, and can be done more concisely
in fact, I posted this a couple of months back and it may prove useful to see a few options for how to create and join threads in clojure:
are those examples also in JCIP? i have never seen it, but intended to do so (i would assume some of them would be, since that’s mostly clojurized java) @U5B9SLY6T
Not examples, exactly. Reading the gist just reminded me of JCIP because it's basically a guide to Java concurrency utils
not only that, but actual work should never be done directly in go blocks, if you need to start a CPU or IO heavy task inside a go block you should use clojure.core.async/thread which returns a channel that you can park on
no, you can block on anything
the point of core.async/thread is that you can park your thread, since it returns a channel
the point of thread is that it returns a channel. you can park on that channel. you put CPU or io heavy ops inside thread, that’s all I’m saying
if you do the cpu or io heavy jobs inside go itself, you can starve the go thread pool, which breaks everything
in fact, I’d assume that code that uses channels inside core.async/thread is probably doing it wrong
Agreed on starving the go thread pool, I just thought referring to parking in a thread block was a little deceptive, because typically that's not what you'd be doing
read it again, I never mentioned parking inside thread
and the whole reason thread exists is so that you can park on it
Yes, agreed on that. I just misread your comment as referring to parking in the thread block, not on the returned chan
@joshjones My needs is starting many threads and waiting all of them finishing and do something with results of each one. So, using the "1 - Clojure futures, and deref to join" have I use (doall (map deref threads))
instead of (run! deref threads)
?
@joshjones That sounds very much like a use for pmap
(parallel map), which maps over items in a collection, but uses a pool of threads to do work in parallel.
One of the factors here is the lifespan of your process; things like pmap and core.async work best for long lived servers, as there's some setup cost for them, and Hotspot needs to get in there and optimize a bit. For something like a startup-and-finish command line tool, manually starting a bunch of one-of threads will probably be better.
I did with pmap like this:
(cp/with-shutdown!
[pool (cp/threadpool (count (util/localidades)))]
(let [result (doall (cp/pmap pool #(validar-localidade %) (util/localidades)))]
(into #{} result)))
with [com.climate.claypoole :as cp]
thread pool and pmap from the same library because the regular pmap has limit from CPU/Core, but with long validar-localidade
sometimes freeze. So, I´m trying to find alternative to something similar.So, I thought using core.async will help me when last function evaluation will close channel to do after work
I prefer to use core.async go-loop in some cases over a for-loop because it is simpler to send a signal to stop the loop. Is that heresy?
if you want a loop, you will definitely have trouble using for, because for is not a loop
@raymcdermott I don't see anything heretical about the snippet you posted
well, depending on what reachable
looks like, obviously. I would've presumed since you're doing network traffic, that your code to check if a server is up would return a channel, which your snippet doesn't really show anything about. Presumably you're eliding whatever that is for demonstration purposes here?
your only other real choice is to use loop/recur
directly...but since you're working with channels here, I'm not sure why you'd want to do that?
What were you thinking of instead when you said for-loop
that makes you think this is heresy?