specter 2024-02-05 | Slack Archive

Hello 👋 I'd like to transform and maybe filter in one go, some thing like this:

(sp/transform [sp/INDEXED-VALS]
                (fn [[i x]] (let [v (some-fn x)]
                              (if (some-pred v) [i v] sp/NONE)))
                [1 2 3 4])

is this possible in specter?

rolt17:02:24

you're not using i ?

Lidor Cohen18:02:01

In what? I return i and v

rolt15:02:19

yes but you're not doing anything with it

rolt15:02:17

maybe use actual function with the expected result ?

rolt15:02:28

because I'm not sure I understand the intent

Lidor Cohen15:02:58

What do you mean? it changes the data. I want it in the overall data for later computations

Lidor Cohen15:02:56

I want to know the original position of each transformed value in case it passes the pred

rolt15:02:05

(sp/transform [sp/INDEXED-VALS] (fn [[i x]] [i x]) [1 2 3 4]) => [1 2 3 4], not [[0 1], [1 2] ...]

rolt15:02:33

oh there's an other path later ?

Lidor Cohen15:02:50

rolt15:02:16

sp/transform [INDEXED-VALS the-filter-transformer somthing-else] ?

Lidor Cohen15:02:04

Currently I can't test that but i think it did return the values with the indices but when I introduced the NONE in order to omit values it throws (something about keyword I'm guessing it tries to do something to NONE)

rolt15:02:30

that's because it would expect [i NONE]

rolt15:02:15

but even then I think it throws

rolt15:02:23

[i nil] would work

rolt15:02:56

now I see the intent but I'm not sure how you would do that

Lidor Cohen15:02:02

My code is performance critical so I'm trying to do the transformation and the filtering in one go. My current implementation is with reduce-kv so if I'm not surpassing that it won't do. If I compare only the transformation without the filtering, specter wins (probably because of transient). Then I tried to add the filtering using NONE and failed

rolt15:02:40

I may have an idea let me try

🙏 1

rolt15:02:28

no can't find it, once you use INDEXED-VALS, when you "leave" you're returning the whole vector. And using view + map-indexed would defeat the purpose

rolt16:02:55

I cooked this abomination: (sp/multi-transform [(sp/view #(map-indexed vector %)) sp/ALL (sp/view (fn [[i x]] [i (some-fn x)])) (sp/if-path (fn [[i x]] (some-pred x)) (sp/terminal identity) (sp/terminal-val sp/NONE))] [1 2 3 4])

Lidor Cohen08:02:45

thank you @U02F0C62TC1 I did some benchmarking on a couple of options and this is what cameout:

(timeit
   10000
   (sp/select [sp/INDEXED-VALS
               (sp/view (fn [[i x]] (let [v (inc x)]
                                      (if (even? v) none [i v]))))
               (sp/pred #(-> % none? not))]
              (vec (range 100)))) := 1150

  (timeit
   10000
   (sp/multi-transform [(sp/view #(map-indexed vector %))
                        sp/ALL
                        (sp/view (fn [[i x]] (let [v (inc x)]
                                               (if (even? v) none [i v]))))
                        (sp/if-path (fn [[i x]] (-> x none? not))
                                    (sp/terminal identity)
                                    (sp/terminal-val sp/NONE))]
                       (vec (range 100)))) := 1725


  (timeit
   10000
   (reduce-kv
    (fn [acc i o]
      (let [v (inc o)]
        (if (even? v)
          acc
          (conj acc [i v]))))
    []
    (vec (range 100)))) := 445

rolt13:02:34

if you're going to use clojure why not use transducers ?

(into [] (comp (map inc) (filter even?)) ...)

Lidor Cohen13:02:04

Might also be perfomant

2024-02-05

Channels