Fork me on GitHub
#core-async
<
2018-04-23
>
noisesmith02:04:40

right, you should avoid doing any blocking operations in a go block, use async/thread instead, and integrate it into your go block via the channel it returns

reefersleep08:04:18

Is there a conceptual difference between

(take! 
  my-chan
 (fn [result]
    (do-stuff-with result))
and
(go
  (do-stuff-with (<! my-chan)))
?

reefersleep08:04:51

Maybe not conceptual difference. The more clear question is perhaps; should I expect these two pieces of code to behave differently?

reefersleep08:04:13

We have an example where the first never seems to do-stuff-with, whereas the second succeeds.

reefersleep08:04:52

This is in cljs, btw

lxsameer09:04:17

@noisesmith alright I see, but what did you mean from "via the channel it returns"

noisesmith15:04:48

async/thread returns a channel, you can use that channel inside a go block to work with the value that came from a long running calculation

noisesmith15:04:31

in my experience, otherwise you end up with a bunch of unneeded complexity writing to channels that don't even need to exist

lxsameer15:04:06

so in my understanding

lxsameer15:04:37

core.async designed for concurrency in mind, not parallelism

lxsameer15:04:56

it does parallel processing but

lxsameer15:04:37

just a simple thread pool approach

noisesmith15:04:59

right, core.async/thread exists because otherwise you could easily starve the primary core.async thread pool, not in order to be an effective parallel processing platform

noisesmith15:04:22

if what you need is to do parallel work extensively you could look into core.reducers or the claypoole library

lxsameer15:04:02

cool are they based on core async ?

lxsameer15:04:04

and what does prevent use to have something that runs a go block on a different thread pool ?

lxsameer15:04:31

for example let's say we write macro called blocking-go

lxsameer15:04:04

which run the content on a different threadpool rather than the main core async thread pool ?

noisesmith16:04:59

it would be a change to channels as well - go blocks transform code into callbacks attached to channels

noisesmith16:04:33

and no, those other libraries are not based on core.async, they are parallelism libraries

lxsameer16:04:55

cools, thanks a lot for the info 🙏

tbaldridge12:04:15

@reefersleep no, those two examples compile down to pretty much the same code

tbaldridge12:04:24

ah, there's one difference though. IIRC there's a small chance that the first example will execute when take! is initially called. Whereas the latter will not start execution until you release control of the main JS thread back to the browser

reefersleep12:04:27

@tbaldridge I’m uncertain by what you mean with “release control”.

reefersleep12:04:01

I’m very interested in hearing more about the difference; we use both methods in our code, and it’d be bad if they sometimes produced different results in ways that we are unaware of.

reefersleep12:04:23

However, it turned out that the exact issue with semantically similar but syntactically different code producing different results resided in other code changes, co-located on that branch and lockstepped with the two above examples.

tbaldridge12:04:27

@reefersleep there are times, when take! will do as much work as it can before releasing control back to the browser. If there's a value available to take off the callback, then the callback will be invoked directly by take! otherwise the callback will be inserted into a queue to run later, and take will return right away. This behavior is configurable: https://github.com/clojure/core.async/blob/master/src/main/clojure/cljs/core/async.cljs#L100

tbaldridge12:04:19

However, by wrapping code in a go you're saying " enqueue this go block into the queue to run later, then return with a channel immediately"

tbaldridge12:04:38

so with take! the callback is sometimes delayed, with a go it's always delayed

reefersleep13:04:01

I think I get it now, @tbaldridge. Though, I haven’t dived into the details of the implementation, and I’m a bit confused. Does what you’re saying mean that on-caller? being truthy ensures synchronous behaviour by take!?

reefersleep13:04:41

Otherwise, I don’t see the difference between the callback being “invoked directly by take! and “inserted into a queue to run later”

reefersleep13:04:25

Really appreciate your help, btw 🙂

tbaldridge14:04:16

@reefersleep yes, it's a optimization for times where the callback is very short lived (the majority of times). And it shouldn't make a difference, but it could complicate situations where other parts of your code has race conditions.

tbaldridge14:04:26

I can't help much more without seeing the rest of the code

reefersleep14:04:24

Like I said, the problem we have has been resolved, so this is really more a matter of me trying to learn something that I can pass on to my colleagues. And I have! 😄

tbaldridge14:04:47

On the second link you can ignore the body of the function, but notice how the first thing it does is call dispatch/run (enqueuing a callback)

reefersleep11:04:36

Cheers, that’s what I was reading from it, too 🙂 And thank you for the dispatch explanation!

tbaldridge14:04:30

The code for dispatch/run is here: https://github.com/clojure/core.async/blob/master/src/main/clojure/cljs/core/async/impl/dispatch.cljs#L31 The dispatcher is an optimized version of setTimeout. It tries to reduce the overhead of switch back to the browser by running as many callbacks as possible in a given amount of time.

👍 4