This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2015-09-03
Channels
- # admin-announcements (241)
- # beginners (53)
- # boot (134)
- # cider (20)
- # clara (3)
- # clojure (170)
- # clojure-argentina (13)
- # clojure-brasil (1)
- # clojure-canada (3)
- # clojure-italy (9)
- # clojure-nl (3)
- # clojure-russia (55)
- # clojurescript (115)
- # code-reviews (18)
- # cursive (8)
- # datomic (14)
- # events (8)
- # hoplon (51)
- # immutant (38)
- # jobs (8)
- # ldnclj (11)
- # melbourne (6)
- # off-topic (2)
- # om (5)
- # onyx (9)
- # re-frame (3)
- # reagent (8)
- # sneer-br (1)
- # sydney (1)
- # testing (14)
teamaverage: there’s something to that… they move all of the work into function composition and keep the data structures out of it
there also ought to be a way to look at them through the lens of a CPS transform which will connect up with other structures which do work like “left associate all the binds"
ha ha lost me : )
... and the form (map-t [rf] (fn [f] ... ) is not the reducer but the transducer?
it lets you compose transducers?
I think I'm getting caught out in what's a transducer and what's plumbing for composing them ...
if I say that a reducer is the shape (value, reduct -> reduct)
does that fit with your intuition?
Yeah, then transucer is (value, reduct -> reduct) -> (value, reduct -> reduct) ?
yeahp
And that way you're talking about input and output values with no mention of collections
What does the "traversing" then? The final thing that takes you pass your composed functiuon into?
when we do that we’ll get a reducer out (value, reduct -> reduct)
which we can call reduce
on to “consume” the input structure
yeah so when you seed the t with r you get your final, composed reducer
and you’ll note that a reducer can be applied to any data structure which “has values” in some order
since you just feed an initial reduct
in, which you need to get from somewhere, but it’ll be determined by your output… maybe it’s an empty vector, or an empty set
Cause after it all, the final reducer still has the signature (value, reduct -> reduct) which (reduce ...) knows how to use ?
Do you still use (reduce .. )?
Oh okay
So I also read about reducers namespace which predates transducers are they solving different things or accomplishing the same things but with different approaches?
I can see the transducers as more reusable as you can just assign your final transduced value to something
Getting away from all those implicit seq/coll that get created if you were to use the usual right -> left nesting
Ah okay
I really like how you can apply them to channels to automatically transform channel values
Saves you having to read from the channel, transform, then put on another channel
so if you apply it to a channel you read off an a
value and write some number, 0 to many, b
values back
Yeahp very nice
I first got into tranducers via channels. I like how they abstract out the collection so it doesn't matter - very generic. Also that they can be composed and eliminate intermediate results, so they should perform nicely.
@meow: "still having to work at them so that they are more familiar" here here. Though I'm also relatively new to Clojure
I still don’t grok transducers. Maybe because most of the examples I’ve seen are too abstract that I can’t see how it can help.
@roberto: do you work with core.async channels at all? If so you might find this interesting:
(defn posmod-sift []
(fn [xf]
(let [seen (volatile! [])]
(fn
([] (xf))
([result] (xf result))
([result input]
(if (every? #(pos? (mod input %)) @seen)
(do (vswap! seen conj input)
(xf result input))
result))))))
(defn chan-of-primes []
(let [inputs (filter odd? (drop 3 (range)))
primes (chan 1 (posmod-sift))]
(put! primes 2)
(onto-chan primes inputs)
primes))
It's from something I wrote here: https://github.com/clojure/core.async/wiki/Sieve-of-Eratosthenes
i’ve only played with clojure in my personal projects. Don’t do anything with it in my day job.
I need to edit it. I used xf
because that's what the official docs use but I prefer to call the reducing function rf
yeah, maybe that is what confuses me when reading about transducers, the tendency to use single char vars
@roberto: go through some of David Nolans core.async channel examples on his site - he makes great use of them. They're great as an abstraction over browser events e.g. mouse clicks and what not
http://swannodette.github.io/2013/07/31/extracting-processes/ is a pretty good post for it
I’ll give that another read. Takes me a while to grok things, mostly only after i’ve found a real world use for it.
Yeah same. I found http://kukuruku.co/hub/funcprog/clojure-transducers-reducers-and-other-stuff quite good for understanding transducers - especially where he explodes them into casual functions
One suggestion that was given to me, which helped a lot, is to look at the source code for some of the core functions that now return transducers, like map
and partition-by
.
"The following functions produce a transducer when the input collection is omitted: map cat mapcat filter remove take take-while take-nth drop drop-while replace partition-by partition-all keep keep-indexed map-indexed distinct interpose dedupe random-sample"
So one way to play with transducers is to just use any of the core functions without specifying a coll and you get back a transducer. Then you can compose them together in different combinations. Then use something like transduce my-comp-of-transducers conj [1 2 3 4 5 whatever]
then if you need a custom transducer, like in the primes example I gave above, you can write a custom one
the power and complexity and unfamiliarity is due to the fact that they operate without regard for the underlying collection (don't know, don't care) and we aren't used to that in most other languages
Is that why if you want a coll at the end, often the last comp of a transducer chain is conj?
Cause usually those functions return collections don't they?
Also, is transduce essentially used to unwrap the xform f
from transducer to reducer and kick of the reduction?
@teamaverage: not sure I follow you
So, (transduce xform f coll)
the xform f
will produce a transducer, right?
Okay, problem solved
will xform not be called with f to make the final reducter?
I'd have to check which gets called with what for the final step - I haven't had a need to worry about that part yet.
Mostly if its a transducer that has state and after every item in the original collection has been processed the transducer needs to be called one final time to flush out whatever it is retaining in its state then that's the chance for it to do so.
So you can go a long way with transducers without having to worry about that part of it.
Transducers do have that magical quality to them where the actual process is hidden from view and, much like middleware, you just specify the things that you want to have happen, but the happening happens automagically. If that makes sense. If you are familiar with middleware or boot tasks it should make sense.
@tel: You mentioned early termination of transducers, and I haven't really worked with that aspect of them. Got any example of how that is useful?
Yeah, I think I'm tripping up not being able to see traversal code
Maybe this is my brain making the imperative -> functional leap ha ha
@teamaverage: Yeah, I think transducers really kick things up even one level higher so you've got more abstraction removing you from imperative code and then you have this sort of middleware-ish "inversion of control"ish aspect as well.
Yeahp, they sure do
The payoff, especially the way they have been interwoven into the existing core code base, is that the same functions that you learn in clojure, like map, filter, partition-by, etc. can not only be used the regular way, but can also provide transducers. So you don't have to learn a whole different set of transducers. You just learn the concept of transducers and then use the transducer forms of all the regular core functions.
Imagine if you had to learn the concept of transducers and then a whole bunch of totally unfamiliar transducers with weird names - no way!
Yeahp, they're tricky enough as it is ; )
Another way to look at transducers is just like the regular function, like say filter odd?
or take 5
that we already know how to use by supplying them with a collection to operate on and instead were saying, hey, give me the ability to take 5 items and filter for the odd ones, but I'm not going to tell you what collection to do that to until later.
And then later, when I tell you the collection, I don't want you to care about the collection specifically, I just want to feed you items and something else will worry about those collection details.
So the transducer-equivalents of map/filter and all that are collection ignorant?
And since you no longer care about the collection details, why don't we just comp
you guys together so you can pass the item from one of you to the other instead of computing a bunch of intermediate results cuz that's just so like '90s inefficient and we're being really functional here.
(just reading the core.map source now)
yeah, so check out map called without a collection:
([f]
(fn [rf]
(fn
([] (rf))
([result] (rf result))
([result input]
(rf result (f input)))
([result input & inputs]
(rf result (apply f input inputs))))))
Yip i see that
Is there a way to print out functions in the REPL?
I know there is cuz I used that to add that feature to devcards, but I never actually use the feature in the repl
So it's late and I'm tired and the closure docs do a better job explaining it but all transducers have that weird but elegant shape of a function that returns a function that has 3 arities for the various parts of the transducing process.
It's mid afternoon here in Australia. Thanks for your help
I'm sure I'll get the lightbulb moment soon enough
Probably just need to try write more transducer code myself instead of readin
What you want to do now is imagine what goes on when you use the transducer returned by map in a specific context, like this:
(transduce (map inc) conj [1 2 3 4])
we don't supply an intial value so conj
will be called and expected to provide one, which it does: []
rf
is conj
, result
is []
, f
is inc
and input will be first 1
then in the next call 2
, then 3
, etc
Yeah, that's the reducer right
f
i mean
well, and rf
so that will look like (conj [] (inc 1))
, then (conj [2] (inc 2))
then (conj [2 3] (inc 3))
Yip, gotcha
Indeed, thanks : )
Ha ha you're probably getting even more out of it than me ; )
to teach is to learn and to learn is to teach
it was a test to make sure I really understand what I'd like to think I finally understand well enough
and I noticed things along the way that I didn't really understand as well as I thought I did, so it was good for me
Indeed
also, I used a transducer today that was a missing piece that I was pulling my hair out over until I found it and, oddly enough, it is brand new to clojure 1.7 and it's just a tranducer, not a modified func that returns a transducer
I'm reading through this, currently up to the "Transformers" section http://kukuruku.co/hub/funcprog/clojure-transducers-reducers-and-other-stuff
cat - https://github.com/clojure/clojure/blob/clojure-1.7.0/src/clj/clojure/core.clj#L7300
Hrm okay, so how do you compose transducers?
(map +) (filter odd?) ??
Oh, that easy
what's kind of cool is that transducers aren't really a thing - they're just functions that play by the rules
I think I'm gettin' it
Yeah, i like they're not some new compiler trickery
Just a new way of thinking
I get blown away by the "meta-ness" of it all ha ha
but once you get it you realize its a pretty small amount of ground that's covered - there isn't that much to learn
I think actually my problem is/was how to get transducers composing
Cause for ages you're talking about reducers of (acc input -> result)
but transducers are ( x -> x')
or whatever
Cause i thought there must be some unwrapping of transducers to get the reducer out ..
I think I just need a break and to let it settle in
I've only been working with clojure for a little over 3 months so I don't really have a history with reducers - I just jumped right into transducers
And you're most likely pulling your hair out ha ha
It's pretty nice to have all your functions in the one place; not spread between parens
What’s the name of the Clojure site that shows usages of a function, libraries dependencies, who uses who, e.t.c.? I always forget the name
clojars?
nope, it’s more interactive than that, it shows source code too
thats it!
Thanks @sofra
has anyone used rethinkdb for streaming changes to web clients? I’m exploring using it in a project and am running into conceptual problems.
@seantempesta: you might ask on the #C073DKH9P and/or #C0620C0C8 channels as there seem to be several folks there that are using rethinkdb
Thanks @meow!
@tel: What I don't understand is when I would need to terminate the reduction early. Got a canonical clarifying example of when one would need that?
then take’s reducer would just be (fn [a r] r)
every time after you’ve passed enough elements
but with early term you know when to stop because you’re never going to affect the result again
just to note that it arises really naturally from the core operation of transducers all together
"Haha, no. I had high hopes for Clojure for a while, but they're fairly user-hostile, even if they think (and loudly assert) that they aren't."
@anthonyrrubin: not sure what would make him say clojure is user-hostile - I've only been using it for 3 months and haven't found that to be the case
I do not know.
I'm not going to lose any sleep over it, that's for sure. Plenty of very helpful people in this community.
Yegge's comment dates back to his interaction with the Clojure community years ago. He had a lot of ideas for improving Clojure, especially with respect to making it more amenable to tooling, all of which were ignored. The discussion basically came down to... Yegge: Clojure is not sufficiently receptive to new ideas. ClojureCommunity: If we added everything that people suggested, the language would suck. Both sides are true.
@anthonyrrubin: he had a lot of nice things to say about clojure in his foreword to Joy of Clojure