This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-06-15
Channels
- # aws-lambda (6)
- # beginners (8)
- # cider (12)
- # cljs-dev (12)
- # cljsrn (9)
- # clojars (2)
- # clojure (161)
- # clojure-berlin (9)
- # clojure-italy (10)
- # clojure-russia (18)
- # clojure-spec (20)
- # clojure-uk (18)
- # clojurescript (37)
- # cursive (13)
- # data-science (1)
- # datomic (23)
- # flambo (2)
- # hoplon (22)
- # immutant (11)
- # jobs (1)
- # klipse (11)
- # leiningen (5)
- # liberator (7)
- # luminus (5)
- # lumo (3)
- # off-topic (28)
- # om (8)
- # onyx (15)
- # parinfer (111)
- # pedestal (21)
- # re-frame (6)
- # reagent (38)
- # rum (4)
- # spacemacs (2)
- # specter (34)
- # unrepl (122)
- # untangled (3)
- # vim (1)
- # yada (8)
Hey, I've got a newbie question. I have a big arbitrarily-nested ball of data (vectors, maps, etc). I want to traverse it looking for maps that pass a certain predicate, and then assoc a new key into those maps.
Here's what my current implementation looks like:
(defn decorate
"walk/postwalk version of above that adds get urls in-place"
[data]
(walk/postwalk
(fn [item]
(if-not (right-thing? item)
item
(my-transform item)))
data))
Can someone advise me on what the equivalent specter calls would be?
@timgilbert it would look something like (setval [(walker right-thing?) :my-new-key] my-new-val data)
or to match your code exactly (transform (walker right-thing?) my-transform data)
depending on what my-transform
is you may be able to express it within the path
It's basically just (assoc my-map :my/key (other-fn my-map))
you can do (transform [(walker right-thing?) (collect-one (view other-fn)) :my/key] (fn [m _] m) data)
Cool. Thanks for the help, I'll play around with it some. I'm benchmarking a few different approaches
the walker
in latest specter release is a lot faster than clojure.walk and has same functionality
i benchmarked 70% improvement
Cool. I'll let you know how it turns out
FWIW, I'm seeing about the same speeds on my test data set, which has a lot of stuff I don't care (about 750K of EDN) about and 0-90 target maps
care to share your benchmark code?
It would take me some time to clean it up and put it into a gist, but I'll try to do it tomorrow afternoon sometime
But it's basically running criterium against a bunch of random data from an EDN file plus a bunch of generated data from specs
The context is I need to go through a potentially large API response and decorate some data with derived values, and I'm trying to see how much of a perf hit I'm setting myself up for
i'd be curious to see benchmark of (transform (walker right-thing?) my-transform data)
approach as well (without using collect-one
)
Looks like about the same. In my benchmarks I've got 0.2927 for the collect-one version, 0.2994 for the transform one and 0.2962 for the postwalk version
(I'm not going absolutely crazy with the methodology here, I should note, just looking for back-of-the-envelope stuff)
if your target maps are never within another map's keys, you can gain big performance improvement by using a custom recursive path
Sadly, they are deeply and somewhat arbitrarily nested
(Which is something I'm looking to fix but that's another story)
(def optimized-walker
(recursive-path [afn] p
(cond-path (pred afn) STAY
map? [MAP-VALS p]
coll? [ALL p]
)))
the data has maps within map keys?
e.g. {[{:a 1}] 2}
Yeah, so it could be like {:company "foo" :people [{:person "bob" :avatar {:type :thumbnail :image-path "/dir/file.png"}}]}
I'm looking for :image-path
But it could also be {:product "foo" :images [{:type :product-img :image-path "foo-bar"}]}
yea, use optimized-walker
...and those things can all nest in various ways
avoiding traversing over key/value pairs and keys should be very significant
Cool. Thanks again for the help. I'll play around with it more tomorrow and let you know if I kick up anything interesting
sure thing