This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-08-20
Channels
- # announcements (1)
- # bangalore-clj (27)
- # beginners (82)
- # boot (4)
- # chestnut (1)
- # cider (22)
- # cljs-dev (26)
- # cljsrn (4)
- # clojure (118)
- # clojure-dev (18)
- # clojure-italy (2)
- # clojure-losangeles (1)
- # clojure-nl (2)
- # clojure-russia (1)
- # clojure-spec (15)
- # clojure-uk (125)
- # clojurescript (61)
- # core-async (74)
- # cursive (2)
- # datomic (41)
- # duct (6)
- # editors (7)
- # emacs (3)
- # events (1)
- # figwheel-main (3)
- # fulcro (111)
- # hoplon (11)
- # jobs-discuss (97)
- # lein-figwheel (99)
- # off-topic (34)
- # onyx (4)
- # parinfer (9)
- # pedestal (4)
- # precept (2)
- # re-frame (5)
- # reagent (2)
- # reitit (4)
- # ring-swagger (11)
- # shadow-cljs (104)
- # spacemacs (4)
- # tools-deps (19)
- # vim (8)
- # yada (15)
@loganpowell Do these execute concurrently?
Blocking/parking take operations coordinate processes. Each pair of output->input processes can coordinate using a channel and blocking/parking take operations. But when execution order is important one must ask whether you need 3 concurrent processes instead of 1.
Hey @bjr! they need to be executed in a specific order
I'm not sure how to answer the concurrency question, but each step needs the value returned from the previous one
I'm still a little fuzzy on the difference between concurrency and parallelism
I've been debugging this for a week and what's currently happening is that all the first step operations are happening, then all the second instead of each process running the three steps then looping
I think it's due to me having nested go
blocks trying to conduct the parking puts/takes
if you want to coordinate the completion of the go block with its calling thread you need to park/block on the channel returned by the go block
Sec, let me put together a quick example
So, for example, what I'm doing is something like this.
The wierd thing is that this works if I use sufficient (<! (timeout...))
s between each function to enable the internal async processes enough time to complete. Otherwise, all of fn1-go->!
functions run, then all of fn2-go->!
yes 😞
might be fine…i’m returning from some time off and am a little fuzzy, can’t remember whether that’s a problem…but why?
In one case, I'm using a library that transforms a promise (returned by a js library) into a <?
take'able, but only in a go
. In others, I'm doing a lot of node fs
interop, which can only handle one operation at a time (Node)
Perhaps I'm over thinking it
I'm currently thinking I should just use put!
(async) for everything except that <?
and pipeline-async
to coordinate the intermediate processes
To give you a sense of what I'm doing, I'm trying to convert about 5000 Census shapefiles to geojson
(for the OSS community)
and you want A to be executing for file2 as soon as it completes for file1, and meanwhile, B begins on file1?
I want all three processes (A->B->C) to happen for file1 before staring the process for file2
but, you're right, what you said is what is actually happening
You might be right
Maybe use futures
or something?
or promessa
How would I do that?
I thought blocking the calling thread is what I needed to do to ensure the order
Obv, I'm still new
(defn do-everything []
(create-folder-structure)
(doseq [f (file-seq ...)]
(save-geojson (zip->geojson f))))
So, one thing is that I have that js library shpjs
, which - unfortunately - returns a js/Promise
I can do that?
I've seen something like this:
(defn test1
"Array approach, flat chain: thread multiple values through promise chain by using Promise.all"
[]
(-> (resolve 5)
(.then square-step)
(.then sum-step)
(.then prn)))
yeah… I think that’s a problem isolated to how you call save-geojson
with the result of zip->geojson
the key being you want to make sure save-geojson
is executed before the next file is loaded
I feel that
the promises stuff is probably there because JS and webworkers — something i’m not too familiar with
i suppose my thought here is get the data out of the promise. you don’t need callbacks so don’t use them.
ah, ok, so how do I coordinate a promise that's nested between other async functions?
(e.g., fs/read
-> promise -> fs/write
)
let me keep digging. Thank you for the help! I have some new thoughts on an approach 🙂
@loganpowell since you are working with promises you might find life easier if you use promesa alet
or async
rather than converting your promises to chan
s repeatedly - http://funcool.github.io/promesa/latest/#async-await-syntax
in js-land i tend to use core.async for operations on streams of things and promesa for promise-based things
@mccraigmccraig Yessir, that's what I'm thinking
I just don't like promises, hence my hesitation... it's like core.async
. Once you introduce promises, those promises dominate the I/O flavor of the code, but - unlike core.async
they're not as composable 😞
I'm thinking about how to modulate the promise code into something that - ultimately - doesn't return a promise, atm (perhaps the interface is a [val chan], but internally uses promises)
promises are perfectly composable! and if you are doing any js i/o then you are already doing promises, so my view is you might as well go with the flow
It's a good point
(which doesn't mean that you shouldn't make as much of your code as possible pure and promise-free)
indeed