This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-11-03
Channels
- # beginners (167)
- # boot (22)
- # chestnut (3)
- # cider (9)
- # clojure (107)
- # clojure-berlin (1)
- # clojure-greece (3)
- # clojure-italy (6)
- # clojure-losangeles (6)
- # clojure-russia (8)
- # clojure-spec (71)
- # clojure-uk (42)
- # clojurescript (186)
- # community-development (1)
- # core-async (12)
- # core-typed (1)
- # css (15)
- # cursive (29)
- # data-science (11)
- # datomic (8)
- # defnpodcast (28)
- # duct (2)
- # fulcro (169)
- # graphql (6)
- # hoplon (3)
- # jobs-discuss (1)
- # kekkonen (5)
- # leiningen (11)
- # lumo (7)
- # off-topic (14)
- # om (1)
- # other-languages (14)
- # portkey (7)
- # re-frame (27)
- # reagent (14)
- # remote-jobs (1)
- # ring-swagger (5)
- # rum (15)
- # shadow-cljs (52)
- # spacemacs (59)
- # specter (78)
- # test-check (3)
- # vim (9)
- # yada (23)
So, this works:
(def data ;; only x and y are allowed
[{:name "x" :rels ["x" "y"]} ;=> fine, keep as is
{:name "y" :rels ["x" "y" "z"]} ;=> keep only allowed rels: {:name "y" :rels ["x" "y"]}
{:name "y" :rels ["z"]} ;=> no allowed rels, remove map completely
{:name "z" :rels ["x" "y"]}] ;=> disallowed name, remove completely
)
(def disallowed? (complement #{"x" "y"}))
(setval
[ALL
(multi-path
[:rels ALL
(pred disallowed?)]
(selected?
(multi-path
[:rels empty?]
[:name (pred disallowed?)])))]
NONE
data)
I was wondering if it’s possible to first express [:name (pred disallowed?)]
, since it might be cheaper to look at the name first. Then it doesn’t have to process the :rels
. E.g.
(setval
[ALL
(multi-path
(selected? :name disallowed?)
[:rels ALL disallowed?]
(selected? :rels empty?))]
NONE
data)
But this triggers a ClassCastException
probably because NONE
is treated as a map or something.When I replace NONE
with {}
it kind of works. Maybe Specter paths should be able to play nicely with NONE
?
This works:
(setval
[ALL
(multi-path
(selected? :name disallowed?)
[#(not= NONE %) :rels ALL disallowed?]
(selected? #(not= NONE %) :rels empty?))]
NONE
data)
(defn SOME? [x]
(not= NONE x))
(setval
[ALL
(multi-path
(selected? :name disallowed?)
[(pred SOME?) :rels ALL disallowed?]
(selected? (pred SOME?) :rels empty?))]
NONE
data)
I quite like the last solution. It almost reads literally like my requirement:
- drop when the name is disallowed
- drop all :rels
that are disallowed
- drop when no :rels
are left
when you remove an element from a sequence, then future navigation in the same path will start at NONE
that's the correct behavior from Specter, and getting a ClassCastException is expected and desired
that solution with SOME?
is fine
you don't need the pred
around SOME?
but you should have it around disallowed?
or define disallowed?
as (pred (complement ...))
I’m not sure why it’s desired? Navigating to NONE
could semantically be treated as a dead end?
actually nvm, you don't need it around disallowed?
since it's defined globally
the element in the sequence was removed, so it's navigated to the "void", represented consistently in Specter as NONE
well, in my ‘real’ code, disallowed is not able to be defined globally, but that’s why I wrapped it in a pred
in the let. Now I don’t understand why I don’t need the pred
around SOME?
, because it’s a global?
multi-path
always executes every subpath in order
SOME?
is statically known to be a function
so specter can figure that out at "compile-time"
ok, so Specter intentionally allows navigating to NONE, so the user can do something with this?
AFTER-ELEM
and BEFORE-ELEM
navigate to NONE
needing that function is very esoteric
btw a better way to implement it is (defn SOME? [v] (not (identical? NONE v)))
yea, avoids some overhead
Hack which makes SOME?
obsolete:
(alter-var-root #'com.rpl.specter.impl/NONE (fn [_] {}))
(setval
[ALL
(multi-path
(selected? :name disallowed?)
[:rels ALL disallowed?]
(selected? :rels empty?))]
NONE
data)
hmm, but this is going to break other stuff probably, because it seems that Clojure uses a singleton object for representing the empty hash-map
yea lol
becomes very hard to set values to empty maps
This could work though:
(def my-NONE {::NONE true})
(alter-var-root #'com.rpl.specter.impl/NONE (fn [_] my-NONE))
(defn SOME? [x]
(not (identical? NONE x)))
;; no SOME? needed
(setval
[ALL
(multi-path
(selected? :name disallowed?)
[:rels ALL disallowed?]
(selected? :rels empty?))]
NONE
data)
(setval AFTER-ELEM 3 [1 2])
(setval BEFORE-ELEM 3 [1 2])
since (identical? {::NONE true} {::NONE true}) ;;=> false
no, it's better to error when trying to navigate on NONE
this code makes NONE
magically "work" for map navigators
but it will still fail for all the other navigators
better to handle it explicitly in code like this
Looking at the vanilla Clojure (which I didn’t write prior to writing Specter), it’s comparable length wise: https://gist.github.com/borkdude/5f9a4ae710217e893a9462ff90b6cac3#file-specter-clj-L20
it changes all the vectors to seqs
true. in this case I didn’t need the vectors much longer as it went out as JSON anyway
length isn't always the best metric of code quality
besides maintaining types, I find the Specter version far more readable
pretty much read left to right instead of having lots of nested expressions
Running a poll… 🙂 https://twitter.com/borkdude/status/926493766441226240
The Clojure version appears to be twice as fast (running 100k times with time, keep
wrapped in doall
, I know, not a robust profiling tool, but still?)
there's probably a bunch of micro-optimizations that can be done
empty?
is slow on vectors
#(= 0 (.length ^IPersistentVector v)) is fastest way I've found
can also change to:
(setval
[ALL
(if-path (selected? :name disallowed?)
STAY
(multi-path
[:rels ALL disallowed?]
(selected? :rels empty?)))]
NONE
data)
so it stops doing work after removing on first condition
#(-> % :name disallowed?)
might be faster
same with #(-> % :rels empty?)
(def my-empty? (fn [v] (= 0 (.length ^clojure.lang.IPersistentVector v))))
(time
(dotimes [i 100000]
(setval
[ALL
(if-path #(-> % :name disallowed?)
STAY
(multi-path
[:rels ALL disallowed?]
#(-> % :rels my-empty?)))]
NONE
data)))
Shaves off roughly 80mssince specter version maintains types and clojure version constructs a different type, it's hard to say how much of difference comes from that
@borkdude the inner call to remove
is also not realizing the full computation
should wrap that in doall
to make comparison fair
@nathanmarz yes, did that, yields about 20ms extra
(defn vanilla []
(doall (keep
(fn [m]
(if-not (disallowed? (:name m))
(if-let [new-rels (doall (seq (remove disallowed? (:rels m))))]
(assoc m :rels new-rels))))
data)))
making this non-lazy (and measured with criterium) Evaluation count : 558918 in 6 samples of 93153 calls.
Execution time mean : 1.115634 µs
Execution time std-deviation : 37.921802 ns
Execution time lower quantile : 1.082254 µs ( 2.5%)
Execution time upper quantile : 1.172033 µs (97.5%)
Overhead used : 1.603741 ns
This specter version: (defn specter-v2 []
(setval [ALL
(if-path #(-> % :name disallowed?)
STAY
(multi-path [:rels ALL disallowed?]
#(-> % :rels empty?)))]
NONE
data))
also measured with criterium: Evaluation count : 344118 in 6 samples of 57353 calls.
Execution time mean : 1.732753 µs
Execution time std-deviation : 12.525124 ns
Execution time lower quantile : 1.716643 µs ( 2.5%)
Execution time upper quantile : 1.744641 µs (97.5%)
Overhead used : 1.603741 ns
so yeah...still faster with the hand-written clojure, but no where near as much as it was when factoring in the fact that one was lazy and the other not before
so...meh. Unless performance is absolutely critical to you, then it's just a style preference here? And if performance is absolutely critical...then always measure very carefully whatever you're doing
@tanzoniteblack That criterium benchmark doesn’t draw a very different picture than what I already had
🙂 wasn't saying my numbers were different then yours, I just couldn't find hard numbers for your final versions to directly compare, so decided to play with it myself
also, I didn't see much of a difference between your custom empty and directly using empty?
when measured with criterium. Don't remember the exact numbers, but only like 50-100 nanoseconds difference with the different empties swapped? Something within the realm of measuring error with the final result being in microseconds, anyways
(just as an FYI)
Oh look, Alex wrote a transducers version too: https://gist.github.com/borkdude/5f9a4ae710217e893a9462ff90b6cac3#gistcomment-2247156