Fork me on GitHub
#specter
<
2017-09-07
>
dadair20:09:43

Would compiling recursive paths give any benefit? I've tried a compiled-transform with comp-paths for the above path vector and don't see any performance benefits (using criterium to get means)

nathanmarz21:09:37

@dadair no that's not necessary anymore

nathanmarz21:09:44

since 0.11.0 specter does inline compilation/caching

nathanmarz21:09:54

feel free to paste your path if you'd like tips on performance

dadair21:09:14

it's the exact path you provided in the code block above. It's improved my app performance quite substantially already, I was just curious of compilation would make it faster as I know the path beforehand

nathanmarz21:09:49

yea that path's performance could be improved

nathanmarz21:09:22

or rather that transform

nathanmarz21:09:31

(def terminal2
  (richnav [afn]
    (select* [this vals structure next-fn]
      (i/throw-illegal "'terminal2' should only be used in multi-transform"))
    (transform* [this vals structure next-fn]
      (afn vals structure))))

nathanmarz21:09:07

(multi-transform [(putval :root) :root nodes-with-path :ref (terminal2 (fn [vals _] vals))] data)

nathanmarz21:09:14

avoids the apply on the transform-fn and the butlast call

dadair21:09:43

is terminal2 on 1.0.3?

nathanmarz21:09:04

I should probably add something like that into a future version

nathanmarz21:09:36

that's a variant of terminal which is in specter

dadair21:09:52

yeah that improved it quite a bit again: prewalk: 27ms, transform: 5.4ms, multi-transform: 1.3ms

dadair21:09:59

this is awesome thank you

nathanmarz21:09:12

that's probably close to optimal

dadair21:09:08

I've also replaced a tree-seq > group-by > get pipeline with:

(def nodes-without-path
  (sp/recursive-path [] p
                     (sp/continue-then-stay
                      (sp/must :children)
                      sp/ALL
                      sp/LAST
                      p)))

(defn find-vals [root type]
  (sp/select [:root nodes-without-path (fn [x] (= type (-> x :descriptor :type)))] root))
any performance tips there?

nathanmarz22:09:10

no, that seems good

dadair22:09:32

also, when the predicate matches, that node will be a leaf, if that adds some performance considerations

nathanmarz22:09:24

oh, you can replace ALL LAST with MAP-VALS

nathanmarz22:09:30

that will be a big performance boost

nathanmarz22:09:20

as for the last property you said, you should be able to change the path to reflect that

nathanmarz22:09:39

basically to only navigate to leaves, not to every intermediate node

nathanmarz22:09:07

depends on how you detect a leaf

nathanmarz22:09:22

non-existence of :children key, or just :children key with empty map?

dadair22:09:38

I was going to say :children with empty map but that's not necessarily true actually, it's more that certain nodes have types (-> node :descriptor :type) that reflect leaf "value" nodes, but internally they may have :children for sub-values

dadair22:09:57

I could check if the type is in a known set of types

nathanmarz22:09:12

it would look like:

nathanmarz22:09:16

(def leaf-nodes
  (recursive-path [] p
    (if-path leaf-node?
      STAY
      [:children MAP-VALS p]
      )))

nathanmarz22:09:22

just fill in leaf-node?

dadair22:09:34

Great I'll give that a try, thanks! Specter is awesome, thanks for open sourcing it!