This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # admin-announcements (241)
- # beginners (53)
- # boot (134)
- # cider (20)
- # clara (3)
- # clojure (170)
- # clojure-argentina (13)
- # clojure-brasil (1)
- # clojure-canada (3)
- # clojure-italy (9)
- # clojure-nl (3)
- # clojure-russia (55)
- # clojurescript (115)
- # code-reviews (18)
- # cursive (8)
- # datomic (14)
- # events (8)
- # hoplon (51)
- # immutant (38)
- # jobs (8)
- # ldnclj (11)
- # melbourne (6)
- # off-topic (2)
- # om (5)
- # onyx (9)
- # re-frame (3)
- # reagent (8)
- # sneer-br (1)
- # sydney (1)
- # testing (14)
teamaverage: there’s something to that… they move all of the work into function composition and keep the data structures out of it
there also ought to be a way to look at them through the lens of a CPS transform which will connect up with other structures which do work like “left associate all the binds"
... and the form (map-t [rf] (fn [f] ... ) is not the reducer but the transducer?
I think I'm getting caught out in what's a transducer and what's plumbing for composing them ...
if I say that a reducer is the shape
(value, reduct -> reduct) does that fit with your intuition?
And that way you're talking about input and output values with no mention of collections
What does the "traversing" then? The final thing that takes you pass your composed functiuon into?
when we do that we’ll get a reducer out
(value, reduct -> reduct) which we can call
reduce on to “consume” the input structure
and you’ll note that a reducer can be applied to any data structure which “has values” in some order
since you just feed an initial
reduct in, which you need to get from somewhere, but it’ll be determined by your output… maybe it’s an empty vector, or an empty set
Cause after it all, the final reducer still has the signature (value, reduct -> reduct) which (reduce ...) knows how to use ?
So I also read about reducers namespace which predates transducers are they solving different things or accomplishing the same things but with different approaches?
I can see the transducers as more reusable as you can just assign your final transduced value to something
Getting away from all those implicit seq/coll that get created if you were to use the usual right -> left nesting
I really like how you can apply them to channels to automatically transform channel values
Saves you having to read from the channel, transform, then put on another channel
so if you apply it to a channel you read off an
a value and write some number, 0 to many,
b values back
I first got into tranducers via channels. I like how they abstract out the collection so it doesn't matter - very generic. Also that they can be composed and eliminate intermediate results, so they should perform nicely.
@meow: "still having to work at them so that they are more familiar" here here. Though I'm also relatively new to Clojure
I still don’t grok transducers. Maybe because most of the examples I’ve seen are too abstract that I can’t see how it can help.
@roberto: do you work with core.async channels at all? If so you might find this interesting:
(defn posmod-sift  (fn [xf] (let [seen (volatile! )] (fn ( (xf)) ([result] (xf result)) ([result input] (if (every? #(pos? (mod input %)) @seen) (do (vswap! seen conj input) (xf result input)) result)))))) (defn chan-of-primes  (let [inputs (filter odd? (drop 3 (range))) primes (chan 1 (posmod-sift))] (put! primes 2) (onto-chan primes inputs) primes))
It's from something I wrote here: https://github.com/clojure/core.async/wiki/Sieve-of-Eratosthenes
i’ve only played with clojure in my personal projects. Don’t do anything with it in my day job.
I need to edit it. I used
xf because that's what the official docs use but I prefer to call the reducing function
yeah, maybe that is what confuses me when reading about transducers, the tendency to use single char vars
@roberto: go through some of David Nolans core.async channel examples on his site - he makes great use of them. They're great as an abstraction over browser events e.g. mouse clicks and what not
http://swannodette.github.io/2013/07/31/extracting-processes/ is a pretty good post for it
I’ll give that another read. Takes me a while to grok things, mostly only after i’ve found a real world use for it.
Yeah same. I found http://kukuruku.co/hub/funcprog/clojure-transducers-reducers-and-other-stuff quite good for understanding transducers - especially where he explodes them into casual functions
One suggestion that was given to me, which helped a lot, is to look at the source code for some of the core functions that now return transducers, like
"The following functions produce a transducer when the input collection is omitted: map cat mapcat filter remove take take-while take-nth drop drop-while replace partition-by partition-all keep keep-indexed map-indexed distinct interpose dedupe random-sample"
So one way to play with transducers is to just use any of the core functions without specifying a coll and you get back a transducer. Then you can compose them together in different combinations. Then use something like
transduce my-comp-of-transducers conj [1 2 3 4 5 whatever]
then if you need a custom transducer, like in the primes example I gave above, you can write a custom one
the power and complexity and unfamiliarity is due to the fact that they operate without regard for the underlying collection (don't know, don't care) and we aren't used to that in most other languages
Is that why if you want a coll at the end, often the last comp of a transducer chain is conj?
Also, is transduce essentially used to unwrap the
xform f from transducer to reducer and kick of the reduction?
I'd have to check which gets called with what for the final step - I haven't had a need to worry about that part yet.
Mostly if its a transducer that has state and after every item in the original collection has been processed the transducer needs to be called one final time to flush out whatever it is retaining in its state then that's the chance for it to do so.
So you can go a long way with transducers without having to worry about that part of it.
Transducers do have that magical quality to them where the actual process is hidden from view and, much like middleware, you just specify the things that you want to have happen, but the happening happens automagically. If that makes sense. If you are familiar with middleware or boot tasks it should make sense.
@tel: You mentioned early termination of transducers, and I haven't really worked with that aspect of them. Got any example of how that is useful?
@teamaverage: Yeah, I think transducers really kick things up even one level higher so you've got more abstraction removing you from imperative code and then you have this sort of middleware-ish "inversion of control"ish aspect as well.
The payoff, especially the way they have been interwoven into the existing core code base, is that the same functions that you learn in clojure, like map, filter, partition-by, etc. can not only be used the regular way, but can also provide transducers. So you don't have to learn a whole different set of transducers. You just learn the concept of transducers and then use the transducer forms of all the regular core functions.
Imagine if you had to learn the concept of transducers and then a whole bunch of totally unfamiliar transducers with weird names - no way!
Another way to look at transducers is just like the regular function, like say
filter odd? or
take 5 that we already know how to use by supplying them with a collection to operate on and instead were saying, hey, give me the ability to take 5 items and filter for the odd ones, but I'm not going to tell you what collection to do that to until later.
And then later, when I tell you the collection, I don't want you to care about the collection specifically, I just want to feed you items and something else will worry about those collection details.
So the transducer-equivalents of map/filter and all that are collection ignorant?
And since you no longer care about the collection details, why don't we just
comp you guys together so you can pass the item from one of you to the other instead of computing a bunch of intermediate results cuz that's just so like '90s inefficient and we're being really functional here.
yeah, so check out map called without a collection:
([f] (fn [rf] (fn ( (rf)) ([result] (rf result)) ([result input] (rf result (f input))) ([result input & inputs] (rf result (apply f input inputs))))))
I know there is cuz I used that to add that feature to devcards, but I never actually use the feature in the repl
So it's late and I'm tired and the closure docs do a better job explaining it but all transducers have that weird but elegant shape of a function that returns a function that has 3 arities for the various parts of the transducing process.
What you want to do now is imagine what goes on when you use the transducer returned by map in a specific context, like this:
(transduce (map inc) conj [1 2 3 4])
we don't supply an intial value so
conj will be called and expected to provide one, which it does:
inc and input will be first
1 then in the next call
so that will look like
(conj  (inc 1)), then
(conj  (inc 2)) then
(conj [2 3] (inc 3))
it was a test to make sure I really understand what I'd like to think I finally understand well enough
and I noticed things along the way that I didn't really understand as well as I thought I did, so it was good for me
also, I used a transducer today that was a missing piece that I was pulling my hair out over until I found it and, oddly enough, it is brand new to clojure 1.7 and it's just a tranducer, not a modified func that returns a transducer
I'm reading through this, currently up to the "Transformers" section http://kukuruku.co/hub/funcprog/clojure-transducers-reducers-and-other-stuff
what's kind of cool is that transducers aren't really a thing - they're just functions that play by the rules
but once you get it you realize its a pretty small amount of ground that's covered - there isn't that much to learn
Cause i thought there must be some unwrapping of transducers to get the reducer out ..
I've only been working with clojure for a little over 3 months so I don't really have a history with reducers - I just jumped right into transducers
It's pretty nice to have all your functions in the one place; not spread between parens
What’s the name of the Clojure site that shows usages of a function, libraries dependencies, who uses who, e.t.c.? I always forget the name
has anyone used rethinkdb for streaming changes to web clients? I’m exploring using it in a project and am running into conceptual problems.
@seantempesta: you might ask on the #C073DKH9P and/or #C0620C0C8 channels as there seem to be several folks there that are using rethinkdb
@tel: What I don't understand is when I would need to terminate the reduction early. Got a canonical clarifying example of when one would need that?
then take’s reducer would just be
(fn [a r] r) every time after you’ve passed enough elements
but with early term you know when to stop because you’re never going to affect the result again
just to note that it arises really naturally from the core operation of transducers all together
"Haha, no. I had high hopes for Clojure for a while, but they're fairly user-hostile, even if they think (and loudly assert) that they aren't."
@anthonyrrubin: not sure what would make him say clojure is user-hostile - I've only been using it for 3 months and haven't found that to be the case
I'm not going to lose any sleep over it, that's for sure. Plenty of very helpful people in this community.
Yegge's comment dates back to his interaction with the Clojure community years ago. He had a lot of ideas for improving Clojure, especially with respect to making it more amenable to tooling, all of which were ignored. The discussion basically came down to... Yegge: Clojure is not sufficiently receptive to new ideas. ClojureCommunity: If we added everything that people suggested, the language would suck. Both sides are true.