Fork me on GitHub
#core-async
<
2015-08-17
>
jthomson11:08:14

Does anyone have a nifty way to drain the contents of a channel into collection in clojurescript? In clojure I just close the channel then loop blocking takes (that don't block because the channel is closed).

jthomson11:08:31

ah, just remembered poll!

ghadi13:08:34

async/into is the function you want

jthomson15:08:18

@ghadi: even better, thanks!

jthomson15:08:56

oh but into also returns a channel.

jthomson15:08:21

poll! works. it's not been released yet but is on master (https://github.com/clojure/core.async/commit/d8047c0)

ghadi15:08:35

Poll isn't for this particular purpose, if I understand you correctly.

ghadi15:08:08

Close the channel, then use into and take from that new channel

jthomson16:08:30

what I want to do is to drain a channel into a collection - specifically so I can store its value and recreate the channel later

meow16:08:12

store what value?

jthomson16:08:12

store the collection of the items left on the channel after it closed

jthomson16:08:58

I'm using this at the moment:

(defn drain!
  "Closes and drains a channel. Synchronously returns a collection of drained messages."
  [chan]
  (async/close! chan)
  (loop [items []]
    (if-let [item (#?(:clj async/<!! :cljs poll!) chan)]
      (recur (conj items item))
      items)))

meow16:08:15

something like this? (def result (go (<! (core.async.into [] my-chan))))

jthomson16:08:23

still returns a channel!

meow16:08:41

oh, right

meow16:08:52

I dealt with that somewhere

jthomson16:08:05

but I could definitely use into to get rid of that loop.

meow16:08:52

I know I've dealt with that exact issue of just getting the darn collection out of the result of core.async.into and it is amazingly frustrating that I cannot remember how. Of course, I'm working in cljs where you can't just do <!!

meow16:08:28

(def result (<!! (core.async.into [] my-chan))) would work in clj, I think

meow16:08:41

just doing this off the top of my head

jthomson16:08:16

yes exactly

meow16:08:31

and take! requires a func

jthomson16:08:42

my uneducated guess was that this was one of the use cases intended for poll!

meow16:08:13

which makes me think of doing a take! and sending it to identity but that makes me think that I had a more elegant solution at some time

jthomson16:08:48

yes I did think about something like that

meow16:08:44

found it

meow16:08:18

@jthomson: so I played around with a game of life simulation using channels

meow16:08:47

if you look at that section of code you'll see some of the ways I dealt with these issues

meow16:08:43

for example, I did this because take needs a func: (take! (step-chan (get-in @state [:gol :cells])) swap-gol!))

meow16:08:51

so that take! is dealing with the channel that gets returned by a go block

meow16:08:50

inside that go block I wanted to return the result of draining a channel, so I end the go block with this:

(let [new-cells (<! (async/into {} generation-ch))]
          (console/time-end "generation-ch")
          new-cells)))))

jthomson16:08:57

yep this makes sense.

jthomson16:08:16

do you have a guarantee that the atom will have been swapped before you deref it?

jthomson16:08:27

i suppose practically if the channel is closed it doesn't matter

meow16:08:32

not sure what you mean

jthomson16:08:52

well you don't know when take! will call that function

jthomson16:08:14

and in my case I want to get the result immediately

meow16:08:20

if you do something like my (let [new-cells (<! (async/into {} generation-ch))] you should be good because you are closing the channel

meow16:08:35

ugh, but that needs to be inside a go block

meow16:08:51

I don't think it's just me. I think there is a missing piece here in the core.async api. We need something like <!! in cljs.

meow16:08:39

@rauh: are you available to look at this?

rauh16:08:10

You will never have blocking takes like <!! available in cljs

rauh16:08:22

since your browser is single threaded and it would freeze the UI

rauh16:08:01

Once you go async in any way the only way to merge back from callback styles things is to end the asynchronicity

rauh16:08:21

Just like the closing of the channel jthomson is doing

meow16:08:47

semantically though it feels like something is missing - if you use <! inside a go block you get back another channel that you then need to take from, which puts you right back in the same boat of having to use take! which requires a function but all we want to do is read the final value from the channel.

rauh16:08:56

There is no other way other than callbacks (which might be sugared by go etc) but there is no way around them as long as you have async code

rauh16:08:39

take! is the right way. (Or poll after closing like jthomson is demonstrating)

meow16:08:16

and using what function with take! - identity?

rauh16:08:49

No, just the function that handles your data.

rauh16:08:44

No way around callbacks, if you're in a single threaded environment. This is why node.js is one giant callback hell and the community has been coming up with hundreds of libaries to somehow avoid those

exupero17:08:32

@meow: I think the function you pass to take! is a callback.

rauh17:08:33

Unless you do fibers (which rewrite stuff in a state machine). But then: That's exactly what our beloved go does

meow17:08:59

but like in the example given by @jthomson where you want a function that takes a channel and returns a collection, without using poll!

meow17:08:31

you know you are going to close the channel, so it won't be an async operation

rauh17:08:04

Correct, then the only way is poll unless you want to tap into implementation details (AFAIK)

meow17:08:07

all you have available in cljs is take! so what function do you give it identity?

rauh17:08:45

No you just give it the function that deals with the data (take! chch (fn [data] (handle-data data))

jthomson17:08:50

if you use take! you still have to 'dosync' the callback somehow.

jthomson17:08:44

atoms are possible, however to illustrate the timing issue:

(-> #(let [!state (atom nil)
           chan (async/chan)]

       ;; Put a value
       (go (async/>! chan %))

       ;; Asynchronously take and swap!
       (async/take! chan (partial reset! !state))

       ;; Deref after some arbitrarily short time
       (async/<!! (go @!state)))
    (map (range 100))
    (doall))

jthomson17:08:04

returns

(nil nil nil 3 nil 5 nil 7 8 9 nil 11 12 13 nil 15 nil nil nil 19 nil 21 22 23 24 25 26 nil 28 nil 30 nil nil 33 34 35 36 nil nil 39 nil 41 42 nil nil 45 nil 47 48 49 nil 51 52 nil 54 55 56 57 nil 59 nil nil 62 63 nil 65 nil 67 68 69 70 nil 72 73 nil 75 76 nil 78 79 80 nil 82 83 84 85 nil nil nil 89 90 nil 92 nil nil nil nil nil 98 99)

jthomson17:08:16

but poll! seems neatest.

jthomson17:08:06

or take! if you can do everything asynchronously

meow17:08:06

Yeah, I guess poll! really is the thing that I conceptually want and I haven't used it so I keep trying to come up with solutions without it, which is dumb because the solution that works the way I want is poll! simple_smile

meow17:08:49

old dogs, new tricks, dinosaur brain, forcing it in...

meow17:08:38

anyhow, game of life using channels was kind of fun, glad I got to revisit that code

jthomson17:08:55

yeah it's a neat project!

rauh17:08:14

There is nothing wrong with doing (and staying there) operations asyncronously. The macros we have in cljs thankfully make this pretty nice IMO

meow17:08:51

@jthomson: thanks - I learned a lot doing it and trying to keep the browser at 60fps

rauh17:08:10

For instance I always use a (dochan [x my-ch] (do-whatever x)) in my async code. Where the dochan is again a go plus some sugar

meow17:08:16

I intentionally didn't optimize the gol algorithm too much because I wanted something that would tax the cpu so I could figure out how to break it up into chunks and yield to the browser often enough to maintain 60fps even though as the gol proceeds it might take longer than one frame to calculate the next generation.

meow17:08:49

gtg bb later

meow17:08:06

@rauh: thanks for the clarity - I knew I could count on you simple_smile

rauh17:08:22

You're welcome. Cheers