This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-06-23
Channels
- # admin-announcements (2)
- # arachne (2)
- # beginners (76)
- # boot (241)
- # cider (14)
- # cljsrn (2)
- # clojars (3)
- # clojure (94)
- # clojure-android (12)
- # clojure-dev (33)
- # clojure-gamedev (1)
- # clojure-greece (3)
- # clojure-india (1)
- # clojure-nl (2)
- # clojure-quebec (3)
- # clojure-russia (21)
- # clojure-spec (38)
- # clojure-uk (72)
- # clojurescript (62)
- # cursive (20)
- # datascript (3)
- # datomic (14)
- # devcards (1)
- # dirac (14)
- # emacs (11)
- # hoplon (7)
- # jobs (2)
- # keechma (1)
- # lein-figwheel (9)
- # leiningen (9)
- # luminus (1)
- # off-topic (6)
- # om (13)
- # onyx (30)
- # planck (181)
- # proton (3)
- # re-frame (6)
- # reagent (6)
- # specter (108)
- # spirituality-ethics (7)
- # untangled (3)
@nathanmarz just saw your post about multi-transform
, looks interesting. Have you considered adding a terminal-val
which sets the value instead of transforms it (like setval
is to transform
)?
@aengelberg: good idea, i'll add that to the branch
@aengelberg: https://github.com/nathanmarz/specter/commit/50d2aa48f57df75b8395950ac0511f63849da400
just realized that setval does not work with collect... is this a bug?
boot.user=> (setval [:a (collect STAY)] 2 {:a 1})
clojure.lang.ArityException: Wrong number of args (2) passed to: impl/compiled-setval*/fn--2959
I was about to say, why not switch constantly
to (fn [_] val)
like setval uses, for efficiency...
ah, just looked at the implementation of constantly
but yea that's a bug
though it's a weird bug to hit
user> (let [f (constantly 1)] (dotimes [i 5] (time (dotimes [i 10000000] (f 2)))))
"Elapsed time: 178.060125 msecs"
"Elapsed time: 164.164117 msecs"
"Elapsed time: 170.549443 msecs"
"Elapsed time: 165.947481 msecs"
"Elapsed time: 163.606619 msecs"
nil
user> (let [f (fn [_] 1)] (dotimes [i 5] (time (dotimes [i 10000000] (f 2)))))
"Elapsed time: 55.36876 msecs"
"Elapsed time: 47.140944 msecs"
"Elapsed time: 46.974314 msecs"
"Elapsed time: 46.51298 msecs"
"Elapsed time: 44.286403 msecs"
nil
user> (let [f (fn [_ & _] 1)] (dotimes [i 5] (time (dotimes [i 10000000] (f 2)))))
"Elapsed time: 86.562498 msecs"
"Elapsed time: 89.103274 msecs"
"Elapsed time: 86.074677 msecs"
"Elapsed time: 87.669437 msecs"
"Elapsed time: 89.690679 msecs"
nil
(edited with more accurate benchmark)There's a spectrum
hand-rolled a fast-constantly
and also fixed that bug
are (transform [<selector> (view f)] identity coll)
and (transform <selector> f coll)
interchangeable? is the latter going to be faster?
concatenative languages can be extremely expressive and allow for easy meta-programming
I played around with combining this idea with destructuring: https://gist.github.com/luxbock/f93529b82792ef35db12fcf5e5a78fdb
@luxbock: they are not interchangeable as f
in the latter form will receive collected values while f
in the former will not
the latter will also be faster
@luxbock: interesting ideas, but can't the starting example be written as this?
(transform
[MAP-VALS
(view frequencies)
(collect-one (subselect MAP-VALS) (view #(apply max %)))
MAP-VALS]
(fn [mx e] (/ e mx))
{:a [1 1 3 4] :b [1 1 2 2 2 3]})
you'll get there đŸ™‚
just realized it can be made even more concise:
(transform
[MAP-VALS
(view frequencies)
(subselect MAP-VALS)
(collect-one (view #(apply max %)))
ALL]
(fn [mx e] (/ e mx))
{:a [1 1 3 4] :b [1 1 2 2 2 3]})
if I wanted to do the same thing, but wanted to inc
all the numbers before using frequencies
and the rest of the selectors, can I somehow fit that into the selector?
I have this type of situation where I had a nested transform
like in my example, and I'm now trying to re-write it using your approach
but instead of numbers in a vector, I have a collection of maps where I need to fetch a nested value and then round those up before calling frequencies
so I need to start with MAP-VALS
, ALL
, (view inc)
, but then I'd need to go back up a level in the selection
@luxbock: is transformed
what you're looking for?
(transform
[MAP-VALS
(transformed ALL inc)
(view frequencies)
(subselect MAP-VALS)
(collect-one (view #(apply max %)))
ALL]
(fn [mx e] (/ e mx))
{:a [1 1 3 4] :b [1 1 2 2 2 3]})
@nathanmarz: yeah it works for my toy example, thanks
my actual use case is still a bit more complicated so I need to figure out a few more things
does view
only take one argument? it seems to accept many but I'm not sure if they are doing anything
it only takes one argument
that it doesn't error with zero or more than one is an implementation artifact
i think it's better to be explicit about that
comp
and view
are completely orthogonal to each other
(view f) (view g) (view h)
is the same as (view (comp f g h))
right? just making sure I understand how it works
i think it would be (comp h g f)
@luxbock: you could use (.intValue (double 12.543))
instead of the string business if that's clojure
@luxbock: you should make the transform function argument for transformed
static – otherwise it can't inline cache
factor it into a defn
@codonnell: yeah good point
that's the only case in specter where anonymous functions to a navigator won't factor automatically
@nathanmarz: ah, why is that?
it would be possible to re-engineer transformed to do so, but that will be a lot of work
is it generally true that I want to do as much of the work in the selector as possible?
I find that's generally more elegant
is there an easier way to do this:
(transform [(collect-one :a :b :c) (view (comp :d :b :a))] + {:a {:b {:c 1 :d 2}}})
?
(reduce + (select [:a :b (multi-path :c :d)] {:a {:b {:c 1 :d 2}}})
or in 0.12.0 (reduce + (traverse [:a :b (multi-path :c :d)] {:a {:b {:c 1 :d 2}}})
don't know if that's easier
I think it's clearer
comp feels unnatural for accessing nested maps, because the keys have to be backwards
one problem that I run into with these large selectors is that they break the cider debugger
that's the inline caching implementation you're seeing
odd that cider would have a limit like that
I benchmarked my specter
implementation of a function vs a transducer / vanillla core version of the same function
results are quite close speed wise but in this case I think I prefer the core version slightly for I think it's easier to go and modify to change its behavior
for example I actually prefer to have the (filter filter-fn)
happen between the two map-steps, and I can't think off the top of my head how I'd need to modify the selector to accomplish this
you probably want a new pathed navigator
selectview
then it would look like:
(defn round-frequencies
[iso-map find-size round-fn filter-fn]
(transform [MAP-VALS
(selectview [ALL (view find-size) (view round-fn) (pred filter-fn)])
(view frequencies)
(transformed
[(subselect MAP-VALS)
(collect-one (view #(apply + %)))
ALL]
revdiv)]
#(into (sorted-map)
(keep (fn [[k v]]
(when (> v 1/100)
[k (* 100 v)])))
%)
iso-map))
it doesn't exist
but it would be easy to make
my third version which uses (transform MAP-VALS ...)
instead of (into {} ...)
is the fastest of them all
If #117 was implemented then it could be simplified further
@luxbock: are you using comp
so your maps and filters are composing as transducers rather than generating intermediate sequences?
rather than ->>
have you checked how much of a performance impact that has?
I'd love to hear the results; I'm curious
@codonnell: specter has a benchmark for that
Benchmark: even :a values from sequence of maps (500000 iterations)
Avg(ms) vs best Code
93.478 1.00 (into [] xf data)
113.85 1.22 (select [ALL :a even?] data)
156.08 1.67 (into [] (comp (map :a) (filter even?)) data)
253.74 2.71 (->> data (mapv :a) (filter even?) doall)
note that comp
with more than two arguments will slow things down a lot
Benchmark: even :a values from sequence of maps (500000 iterations)
Avg(ms) vs best Code
155.98 1.00 (into [] (comp (map :a) (filter even?)) data)
164.82 1.06 (into [] xf data)
203.00 1.30 (select [ALL :a even? even? even?] data)
331.59 2.13 (into [] (comp (map :a) (filter even?) (filter even?) (filter even?)) data)
525.34 3.37 (->> data (mapv :a) (filter even?) (filter even?) (filter even?) doall)
here xf
is (comp (map :a) (filter even?) (filter even?) (filter even?))
wow, that's a huge slowdown
https://github.com/clojure/clojure/blob/clojure-1.7.0/src/clj/clojure/core.clj#L2426
it slows down because it does a reduce after more than two args
if the implementation of comp
was unrolled up to 20 arguments it would work much better
interesting
I wonder why it isn't unrolled like that; other functions in core certainly are.
there's a bunch that aren't: http://dev.clojure.org/jira/browse/CLJ-731
Perhaps they've assumed that people aren't going to be calling comp
over and over like in the benchmark.
well it seems like with transducers it would be a common pattern to do it inline like that
that's true, though transducers weren't in existence when that patch was submitted
@nathanmarz It seems like the multi-transform
functionality of doing a variety of operations in a single transform is still "doable" with transform
, if you collect values along the way and use that to dispatch on some operation in the transform fn.
it's probably not as performant as multi-transform but might be fun to add to your benchmark.
@aengelberg: true but it's a bit convoluted and definitely would be less performant
the cases I consider for the benchmarks are the fastest impls, the most concise impls, and the most idiomatic impls