Fork me on GitHub
#core-async
<
2015-08-12
>
exupero13:08:23

@rauh Thanks. Works in Clojure. Unfortunately, it doesn't in ClojureScript due to the use of <!!.

exupero13:08:12

Actually, my example yesterday with 'into', 'merge', and 'map' does work.

rauh13:08:48

@exupero: You can also just warp it in a go then if you don't have the <!! in cljs

exupero13:08:58

Indeed. Very nice! Thanks for the tip.

oliy14:08:23

hi all, found something weird with transducers and core.async

oliy14:08:51

the transducer function is not used if the buffer is full (which happens all the time if you are using a fixed buffer of 0)

oliy14:08:06

it's very unexpected, is it a bug?

oliy14:08:10

(def c (chan 1 (map inc)))
(onto-chan c [1 2 3])
(<!! c) ;; 2

oliy14:08:22

with a fixed buffer of 1 it is transformed as i expect

oliy14:08:46

(def c (chan 0 (map inc)))
(onto-chan c [1 2 3])
(<!! c) ;; 1

oliy14:08:00

with a fixed buffer of 0 it is not transformed

erik_price15:08:49

transducers require a buffer of at least 1

oliy15:08:18

do you know why?

rauh15:08:30

https://github.com/clojure/core.async/blob/d073896192fa55fab992eb4c9ea57b86ec5cf076/src/main/clojure/clojure/core/async.clj#L82 I think this should be "fixed" to (and buf-or-n (pos? buf-of-n)), silently ignoring it is probably not the best thing to do

oliy15:08:19

you could still call (fixed-buffer 0) - it's not always a number

rauh15:08:09

Edit: Nvm

oliy15:08:18

from looking at the implementation, even if you had a buffer that was not size zero it would still ignore your transducer if the buffer was full

oliy15:08:28

although i haven't been able to write code to prove that yet

rauh15:08:54

But I think best would be a (pos? (count buf)) after making it a buffer, that'd cover all

oliy15:08:24

yes, that would stop you falling into this trap

oliy15:08:00

i still don't understand why transducer transforms are ignored when you don't have a buffer though

erik_price15:08:18

As far as I have ever been able to tell, it’s an implementation detail. I’ve looked at the code, and it’s clear that if you want to use a xducer that it requires a buffer. I’ve never heard a satisfactory explanation for this design, though I assume it exists.

oliy15:08:18

yes, the transducing fn is called when you write into the buffer

oliy15:08:50

it must have been a conscious design because of the guard in the chan function, but i'd like to know the rationale as well

oliy15:08:03

thanks for your thoughts

erik_price15:08:17

I asked this question once in IRC

erik_price15:08:09

[12:04:40] crash_ep: Does anyone know the reason a buffer must be explicitly provided when a channel is created with a transducer?
[12:06:02] crash_ep: (in CLJS)
[12:20:13] dnolen: crash_ep: that's just how it works, you have to transduce on something, buffer size 1, true for Clojure too
[12:21:23] crash_ep: dnolen: maybe I'm misunderstanding what a "buffer" is… I assumed `(chan)` creates a buffer of size 0.
[12:21:47] dnolen: crash_ep: semantically unbuffered
[12:22:17] crash_ep: dnolen: right. Why shouldn't a transducer operate on values put on a semantically unbuffered channel?
[12:22:25] dnolen: crash_ep: no
[12:22:32] dnolen: it has to transduce on *something*
[12:24:01] crash_ep: dnolen: can you help me better understand what "transduce on" means? I thought transduction was an operation on values, but it sounds like something more nuanced than that.
[12:24:27] dnolen: crash_ep: sorry, there are plenty of implementations you can read over now
[12:24:40] crash_ep: dnolen: ok, i'll take another look
[12:24:41] dnolen: the JS one provided by Cognitect that I wrote is particularly short
[12:24:58] dnolen: crash_ep: 
[12:25:44] crash_ep: I guess the part I'm not getting is the relationship to buffer size.
[12:26:12] dnolen: crash_ep: just replace "buffer" with "empty list"
[12:26:14] dnolen: doesn't make any sense
[12:30:49] crash_ep: dnolen: I thought transducers operated on the values as they pass through a channel, not a buffer.
[12:31:03] dnolen: crash_ep: nope
[12:31:08] crash_ep: But it sounds like you're saying that a transducer is a property of a buffer, not a channel
[12:31:21] dnolen: crash_ep: transducers work on buffers yes
[12:31:35] crash_ep: then why don't we pass transducers to the `buffer` function instead of the `chan` function?
[12:33:13] dnolen: crash_ep: that doesn't many any sense either
[12:33:34] dnolen: crash_ep: the channel *uses* the buffer, it knows when and how to apply the transformer
[12:33:45] dnolen: s/many/make
[12:33:59] tbaldridge: dnolen: but the channel basically wraps the .add method of the buffer with the transducer,
[12:35:36] tbaldridge: so you could technically do it in the buffer, but I don't know that it would be any better than putting it in the channel.
[12:36:15] dnolen: tbaldridge: right which was my point, and less flexible if for some reason down the line something else needs to happen, encapsulation stinks blah blah blah :)
[12:38:06] crash_ep: The relationship between these three things is not entirely clear to me, I'll have to study some more. Thanks dnolen tbaldridge

oliy15:08:38

that's left me as confused, possibly more confused, than i was before

oliy15:08:48

i think the arguments apply more to something like reducing than mapping or filtering

rauh15:08:07

I think the problem is that the transducers don't even respect the buffer size. So if a transducer produces multiple elements then it might make the buffer hold more elements than it's supposed to

rauh15:08:15

or the other reason could be that an unbuffered channel will always park unless somebody has already indicated a take on the channel. Which means that a transducer couldn't potentially work. It could never produce more elements

rauh15:08:26

Just guesses though

oliy15:08:15

that second point makes sense, but if it wanted to produce more than the size of the buffer it would still block

oliy15:08:40

i might try seeing what happens if a transducer produces more elements than the buffer can hold

oliy15:08:13

hmm, it just works

oliy15:08:00

(def c (chan 1 cat))
(>!! c [1 2 3])
(<!! c) ;; 1
(<!! c) ;; 2
(<!! c) ;; 3

oliy15:08:25

so there were 3 things in the buffer even though it only had a size of 1

oliy16:08:09

i understand the buffer has to at least exist for this to happen, but if the limit can be exceeded by the transducer then the same could be true of a buffer of fixed size 0

oliy16:08:33

ah great thanks, none of my searches picked that up