Fork me on GitHub
#specter
<
2017-12-19
>
mbjarland16:12:54

@nathanmarz apropos our conversation about warnings under clojure 1.9.0, a new version of midje has been released with an updated (direct) dep on specter:

[midje "1.9.1" :scope "test"]
   [com.rpl/specter "1.0.4" :scope "test" :exclusions [[org.clojure/clojure] [org.clojure/clojurescript]]]
     [riddley "0.1.12" :scope "test"]
the warnings are gone and the related issue on midje (https://github.com/marick/Midje/issues/427) closed

rafael17:12:28

Hi. Don't know if this is the right forum, I have a newbie specter question.

rafael17:12:15

I wanted to filter out elements not matching the path for a transform call.

rafael17:12:38

It's easier to explain in code. The following (specter/transform [specter/ALL (specter/must :a) (specter/must :b)] str [{:a {:b 10}} {:a {:z 42}}]) returns [{:a {:b "10"}} {:a {:z 42}}]

rafael17:12:00

It called the transformation function for an element matching the path and returned the rest unchanged.

rafael17:12:27

I'm wondering if is there anything I can do to make a similar call return [{:a {:b "10"}}] instead?

rafael17:12:53

(filtering out the non-matching {:a {:b 10}} element)

tanzoniteblack17:12:14

@rafael if you want to do a select and a transform in the same operation, you probably want to do select to get only the value you want returned and then use the view selector to make your transformation (https://github.com/nathanmarz/specter/wiki/List-of-Navigators#view)

tanzoniteblack17:12:45

i.e. how I think of it: transform is for modifying in place a piece of a structure; select is for pulling out specific information I want from a structure; and select with the view selector is for doing both selecting only a piece and applying a transformation

rafael17:12:20

Thanks @tanzoniteblack. I'm trying it out now

rafael17:12:23

Naively applying view to the example above gets me:

(specter/select [specter/ALL (specter/must :a) (specter/must :b) (specter/view str)] [{:a {:b 10}} {:a {:z 42}}])
=> ["10"]

rafael17:12:51

I wonder if is there a way to get [{:a {:b "10"}}] in the result.

rafael17:12:41

It would perfectly fine to chain a call to select and a another to transform, or whatever, but even then I'm having a hard time coming up with the right incantation 🙂

tanzoniteblack17:12:58

not actually sure off the top of my head, sorry

rafael17:12:42

No worries, thanks for the help. I'm checking out transformed now

tanzoniteblack17:12:20

(specter/select [(specter/filterer (specter/must :a) (specter/must :b)) 
                                            specter/ALL
                                            (specter/transformed [:a :b] str)]
                                           [{:a {:b 10}} {:a {:z 42}}])
@rafael that appears to do what I think you want?

tanzoniteblack17:12:05

filterer removes all items that don't have :b key inside the value at the :a key, and then for each item in that list, we're going to reach into the path :a then :b and run the function str

tanzoniteblack17:12:18

probably not more efficient then the manual way of doing it? But no idea

rafael17:12:28

The output looks right. I'm trying to digest it before applying to my real use case (which deals with a structure a bit more complicated than the example). Efficiency is not super important for my use case, I'm reaching for specter for the terseness in dealing with a deeply nested structure, so it should be alright

nathanmarz18:12:14

@tanzoniteblack you're looking for not-selected?

nathanmarz18:12:22

(setval [ALL (not-selected? (must :a :b))] NONE data)

rafael18:12:21

With not-selected and setval the complete example would then be:

(->> data
       (S/setval [S/ALL (S/not-selected? (S/must :a :b))] S/NONE)
       (S/transform [S/ALL (S/must :a :b)] str))
?

nathanmarz18:12:30

yea, that would work

nathanmarz18:12:53

for second part you don't need must anymore since they're guaranteed to be there by first part

rafael18:12:47

Cool, now off to apply it to the real use case 🙂

rafael19:12:26

Hi again. When applying to my actual use case things turned out to be more complicated. There are multiple S/ALL where we want to remove non-matching elements. Again, a code example is probably easier to explain:

(def data [{:a [{:b 10}]} {:a [{:b 20} {:z 42}]}])

  (->> data
       (S/setval [S/ALL (S/not-selected? (S/must :a) S/ALL (S/must :b))] S/NONE)
       (S/transform [S/ALL :a S/ALL :b] str))
results in
=> [{:a [{:b "10"}]} {:a [{:b "20"} {:z 42, :b ""}]}]
but I wanted [{:a [{:b "10"}]} {:a [{:b "20"}]}]

rafael19:12:52

(it's a similar example to above, but with one extra nested sequence)

rafael19:12:16

I could get the output I wanted with an extra setval:

(->> data
       (S/setval [S/ALL (S/not-selected? (S/must :a) S/ALL (S/must :b))] S/NONE)
       (S/setval [S/ALL (S/must :a) S/ALL (S/not-selected? (S/must :b))] S/NONE)
       (S/transform [S/ALL :a S/ALL :b] str))

rafael19:12:28

This works, but I was wondering if there is any way of reducing the duplication

nathanmarz19:12:47

@rafael (setval [ALL :a ALL (transformed (must :b) str) (not-selected? (must :b))] NONE data)

nathanmarz20:12:14

do you also want to remove top level maps that don't have any inner maps under :a with :b?

rafael20:12:00

Ideally yeah, If data is (def data [{:a [{:b 10}]} {:a [{:b 20} {:z 42}]} {:a [{:z 42}]}]) then I also need to remove the last {:a [{:z 42}]}

rafael20:12:13

(trying to understand your suggestion above... thank's for taking the time to help)

nathanmarz20:12:33

@rafael this does the trick:

(defdynamicnav with-matching [path]
  (if-path path
    path
    (terminal-val NONE)
    ))

(transform
  [ALL
   (with-matching (must :a))
   ALL
   (with-matching (must :b))
   ]
  str
  data)

nathanmarz20:12:11

even better:

nathanmarz20:12:15

(defdynamicnav with-matching [path]
  (if-path path
    (multi-path path (if-path path STOP (terminal-val NONE)))
    (terminal-val NONE)
    ))

(defdynamicnav with-non-empty [path]
  (multi-path path (if-path empty? (terminal-val NONE)))
  )

(transform
  [ALL
   (with-matching (must :a))
   (with-non-empty ALL)
   (with-matching (must :b))
   ]
  str
  data)

nathanmarz20:12:07

with [{:a [{:b 10}]} {:a [{:q 20} {:z 42}]}] that removes the second map completely

nathanmarz20:12:59

they're similar to macros

rafael20:12:46

Looks like I have a lot of reading to do. Thanks

nathanmarz20:12:18

actually this is a more flexible implementation I think:

(defdynamicnav with-matching [path]
  (if-path path
    (multi-path path (if-path path STOP (terminal-val NONE)))
    (terminal-val NONE)
    ))

(defn ^:direct-nav ensure-pred [pred-fn]
  (path (stay-then-continue (if-path (pred pred-fn) STOP (terminal-val NONE)))))

(transform
  [ALL
   (with-matching (must :a))
   (ensure-pred #(-> % empty? not))
   ALL
   (with-matching (must :b))
   ]
  str
  data)

nathanmarz20:12:41

I may add some variant of these into specter

nathanmarz20:12:11

I think this is the simplest solution:

(defdynamicnav ensure-matching* [path]
  (multi-path path (if-path path STOP (terminal-val NONE))))

(def ensure-matching (eachnav ensure-matching*))
    
(transform
  [ALL
   (ensure-matching (must :a) ALL (must :b))
   ]
  str
  data)