Fork me on GitHub

@rauh Thanks. Works in Clojure. Unfortunately, it doesn't in ClojureScript due to the use of <!!.


Actually, my example yesterday with 'into', 'merge', and 'map' does work.


@exupero: You can also just warp it in a go then if you don't have the <!! in cljs


Indeed. Very nice! Thanks for the tip.


hi all, found something weird with transducers and core.async


the transducer function is not used if the buffer is full (which happens all the time if you are using a fixed buffer of 0)


it's very unexpected, is it a bug?


(def c (chan 1 (map inc)))
(onto-chan c [1 2 3])
(<!! c) ;; 2


with a fixed buffer of 1 it is transformed as i expect


(def c (chan 0 (map inc)))
(onto-chan c [1 2 3])
(<!! c) ;; 1


with a fixed buffer of 0 it is not transformed


transducers require a buffer of at least 1


do you know why?

rauh15:08:30 I think this should be "fixed" to (and buf-or-n (pos? buf-of-n)), silently ignoring it is probably not the best thing to do


you could still call (fixed-buffer 0) - it's not always a number


Edit: Nvm


from looking at the implementation, even if you had a buffer that was not size zero it would still ignore your transducer if the buffer was full


although i haven't been able to write code to prove that yet


But I think best would be a (pos? (count buf)) after making it a buffer, that'd cover all


yes, that would stop you falling into this trap


i still don't understand why transducer transforms are ignored when you don't have a buffer though


As far as I have ever been able to tell, it’s an implementation detail. I’ve looked at the code, and it’s clear that if you want to use a xducer that it requires a buffer. I’ve never heard a satisfactory explanation for this design, though I assume it exists.


yes, the transducing fn is called when you write into the buffer


it must have been a conscious design because of the guard in the chan function, but i'd like to know the rationale as well


thanks for your thoughts


I asked this question once in IRC


[12:04:40] crash_ep: Does anyone know the reason a buffer must be explicitly provided when a channel is created with a transducer?
[12:06:02] crash_ep: (in CLJS)
[12:20:13] dnolen: crash_ep: that's just how it works, you have to transduce on something, buffer size 1, true for Clojure too
[12:21:23] crash_ep: dnolen: maybe I'm misunderstanding what a "buffer" is… I assumed `(chan)` creates a buffer of size 0.
[12:21:47] dnolen: crash_ep: semantically unbuffered
[12:22:17] crash_ep: dnolen: right. Why shouldn't a transducer operate on values put on a semantically unbuffered channel?
[12:22:25] dnolen: crash_ep: no
[12:22:32] dnolen: it has to transduce on *something*
[12:24:01] crash_ep: dnolen: can you help me better understand what "transduce on" means? I thought transduction was an operation on values, but it sounds like something more nuanced than that.
[12:24:27] dnolen: crash_ep: sorry, there are plenty of implementations you can read over now
[12:24:40] crash_ep: dnolen: ok, i'll take another look
[12:24:41] dnolen: the JS one provided by Cognitect that I wrote is particularly short
[12:24:58] dnolen: crash_ep: 
[12:25:44] crash_ep: I guess the part I'm not getting is the relationship to buffer size.
[12:26:12] dnolen: crash_ep: just replace "buffer" with "empty list"
[12:26:14] dnolen: doesn't make any sense
[12:30:49] crash_ep: dnolen: I thought transducers operated on the values as they pass through a channel, not a buffer.
[12:31:03] dnolen: crash_ep: nope
[12:31:08] crash_ep: But it sounds like you're saying that a transducer is a property of a buffer, not a channel
[12:31:21] dnolen: crash_ep: transducers work on buffers yes
[12:31:35] crash_ep: then why don't we pass transducers to the `buffer` function instead of the `chan` function?
[12:33:13] dnolen: crash_ep: that doesn't many any sense either
[12:33:34] dnolen: crash_ep: the channel *uses* the buffer, it knows when and how to apply the transformer
[12:33:45] dnolen: s/many/make
[12:33:59] tbaldridge: dnolen: but the channel basically wraps the .add method of the buffer with the transducer,
[12:35:36] tbaldridge: so you could technically do it in the buffer, but I don't know that it would be any better than putting it in the channel.
[12:36:15] dnolen: tbaldridge: right which was my point, and less flexible if for some reason down the line something else needs to happen, encapsulation stinks blah blah blah :)
[12:38:06] crash_ep: The relationship between these three things is not entirely clear to me, I'll have to study some more. Thanks dnolen tbaldridge


that's left me as confused, possibly more confused, than i was before


i think the arguments apply more to something like reducing than mapping or filtering


I think the problem is that the transducers don't even respect the buffer size. So if a transducer produces multiple elements then it might make the buffer hold more elements than it's supposed to


or the other reason could be that an unbuffered channel will always park unless somebody has already indicated a take on the channel. Which means that a transducer couldn't potentially work. It could never produce more elements


Just guesses though


that second point makes sense, but if it wanted to produce more than the size of the buffer it would still block


i might try seeing what happens if a transducer produces more elements than the buffer can hold


hmm, it just works


(def c (chan 1 cat))
(>!! c [1 2 3])
(<!! c) ;; 1
(<!! c) ;; 2
(<!! c) ;; 3


so there were 3 things in the buffer even though it only had a size of 1


i understand the buffer has to at least exist for this to happen, but if the limit can be exceeded by the transducer then the same could be true of a buffer of fixed size 0


ah great thanks, none of my searches picked that up