Fork me on GitHub
#specter
<
2016-05-31
>
diba01:05:14

pardon my noobness, but i’m just starting out w/ specter, and i’m guessing there’s a better way to do the following that achieves better performance, and would love any guidance:

diba01:05:21

actually, here’s a simpler example where i’m still seeing my (likely non-optimal) specter code taking twice as long:

diba01:05:46

and "f3 #(reduce-kv (fn [m k v] (assoc m k (dec v))) {} %)” seems to be twice as fast as f1

luxbock05:05:36

ALL FIRST and ALL LAST seem like really common selectors, @nathanmarz what do you think about including these as KEYS and VALS?

luxbock06:05:20

or perhaps you could have another namespace that contains functions named after the ones in clojure.core but which work as selectors

nathanmarz13:05:40

@diba @luxbock Yea the slowdown is caused by it just using generic functions to access the map as a sequence

nathanmarz13:05:29

and coercing the MapEntry's to vectors probably wasn't the right thing to do in retrospect, KEY and VAL selectors would be faster than coercing and using FIRST and LAST

nathanmarz13:05:26

I'll look into using reduce-kv for the map case of ALL, but otherwise you can make specialized navigators to make it faster

nathanmarz13:05:56

as for having KEYS and VALS as common navigators, maybe

nathanmarz13:05:27

so far I've been hesitant to include compound navigators in the core library (since there's infinite of them possible), but I can see the argument in this case

nathanmarz14:05:49

Big release for Specter today with the new inline factoring and caching feature

nathanmarz14:05:04

No need to do manual precompilation anymore to get high performance

nathanmarz14:05:35

Also wrote up a guide to the internals of how Specter achieves high performance, recommended reading for all users: https://github.com/nathanmarz/specter/wiki/Specter-0.11.0:-Performance-without-the-tradeoffs

luxbock15:05:54

@nathanmarz: transform seems to have disappeared from com.rpl.specter in the new version

luxbock15:05:06

as well as select

nathanmarz15:05:35

it's in the macros namespace

nathanmarz15:05:15

I recommend reading the changelog for the release as there were a few other backwards incompatible changes

tony.kay16:05:05

I'm wanting to walk a recursive data structure composed of maps and vectors. When inside of a map I'd like to remove k-v pairs where the value matches a predicate. Can I do that with specter? { :a 1 :b [{:x 4 :y 3}] } ---remove where v == 4 --> {:a 1 :b [{:y 3}]}

tony.kay16:05:23

I can easily nullify the value..seems like collect...sort-of: on the k-v pair instead of the value

nathanmarz16:05:43

@tony.kay: in that example do you mean remove x when x==4?

tony.kay16:05:12

remove the :x 4 key-value pair when the value is 4, independent of the key

tony.kay16:05:35

by v I meant the value of a key-value pair

nathanmarz16:05:54

Oh I see remove all key/value pairs where the value matches some predicate

tony.kay16:05:38

perhaps a walker on map?

tony.kay16:05:26

with a nested transform

nathanmarz16:05:50

there's a couple ways to do it

nathanmarz16:05:02

one way is to do something like (transform [:b (collect ALL (selected? LAST #(= % 4)) FIRST)] (fn [keys m] (apply dissoc m keys)))

nathanmarz16:05:02

oh you want it to be recursive as well

nathanmarz16:05:49

one sec, I'll show you

nathanmarz16:05:59

(declarepath RecursiveMaps)

(providepath RecursiveMaps
  [(walker map?)
   (continue-then-stay
     ALL
     LAST
     RecursiveMaps
     )])

(transform
  [RecursiveMaps
   (collect ALL (selected? LAST #(= % 4)) FIRST)
   ]
   (fn [keys m] (apply dissoc m keys))
   {:a 4 :z 11 :b [{:x 4 :y 3} {:a 1}]}
  )
;; => {:z 11, :b [{:y 3} {:a 1}]}

tony.kay16:05:20

oh boy...would not have gotten that 🙂

tony.kay16:05:06

but cool nonetheless!

nathanmarz16:05:21

this is more advanced Specter usage, for sure

nathanmarz16:05:31

but built on the same principles

tony.kay16:05:46

Yeah, I understand it...just not able to write it yet

nathanmarz16:05:31

yea, my first few recursive navigations took me awhile even though I came up with this technique

nathanmarz16:05:46

now I don't even think about it

tony.kay16:05:27

I appreciate the help. I'll read through these functions and maybe play on paper a bit

tony.kay17:05:34

I see select-first, and it mentions it is not more efficient....is there a way of detecting the presence of a value in a nested data structure without continuing a recursive walk?

nathanmarz17:05:29

like do a recursive search for a value matching a predicate and stop once it's reached?

nathanmarz17:05:05

you can do that for a lot of cases with com.rpl.specter.zipper

nathanmarz17:05:19

user=> (transform [z/VECTOR-ZIP (z/find-first #(and (number? %) (even? %))) z/NODE] inc [1 3 5 [2 4]])
[1 3 5 [3 4]]

tony.kay17:05:11

and you're just using inc because you need something

tony.kay17:05:31

oh...nvm, I see

nathanmarz17:05:33

oh I was just showing it for the transform case

nathanmarz17:05:49

works as you'd expect for select