Fork me on GitHub
#specter
<
2017-11-29
>
plamen09:11:47

Hello Nathan, 1. Specter is an amazing library, thank you for providing it! 2. Couple of questions around recursion: In the following map, I do currently a lot of transformations using reduce/reduce-kv, loops, assoc-in/update-in etc., but as I started to rewrite the manipulations using Specter, the code got of course more readable and consistent. {:name ’A :info ... :parameters ... :backend {:package-name “a”} :inputs {’IA {:connector :csv :endpoint ... :variables {’iax {:name ’ix :type :double :rank 0} ’iay {:name ’iy :type :double :rank 2}}} ’IB {:... ... :variables {’ibx {...} ’iby {...}} }} :variables {’Avx { ’Avy {...} ’Avz {:name “Avz” :backend {:package-name “a.avz” :array-layout :column-major :array-memory :off-heap} :type :record :variables {’Avzx {...} ’Avzy {...} ’Avzz {... :name ’Avzz :backend {:package-name “a.avz.avzz”}}}}}} Now - in recursion scenarios like in adding a key :name with value the key of the map itself (where select result and transformation target are the same) like input: :variables {’iax {:type :double :rank 0} ’iay {:type :double :rank 2}} output: :variables {’iax {:name ’ix :type :double :rank 0} ’iay {:name ’iy :type :double :rank 2}} everything works, but I probably miss some documentation or understanding to do recursion involving values from nodes from an upper or dispatching on values from a lower level. Examples: - how do I create recursively the entry :backend {:package-name ???} - where ??? is the path of :name to the selected/trasfomed map (as in the manually added package names in ’A ’Avz and ’Avzz) - while in this case not touching :variables maps starting at the :inputs key in one case? - or transforming both :variables paths (starting at :variables AND :inputs)? - is there a way to collect on the go the current depth of recursion in Specter (or path?) in that same transformation query? - is there a way to dispatch (or do separate transformation over the structure), but whose input can be in higher level than the tranformation target (e.g. can I select/transform the variable ’Avzx based on a :backend value in ’Avz)? - is there a way to dispatch (or do separate transformation over the structure), but whose input can be in lower level than the tranformation target (e.g. can I select/transform the variable ’Avz based on a [:backend :array-layout] and [:backend :array-memory] values in ’Avz)? In all the questions my main problem is less how to solve it outside of Specter, but how to select and recurse where selection/collection come from other nodes that the transformation target.

nathanmarz14:11:26

by collecting a path as you go like that, you can compute the recursion depth

nathanmarz15:11:44

if you use a multi-transform, you could dispatch the transformation function like:

(if-path [:backend (pred= :a)]
  (terminal transformer1)
  (terminal transformer2))

nathanmarz15:11:30

I'm not sure what you mean by your last question, it would help if you simplified your example and showed desired input/output

plamen15:11:35

Hello Nathan, a lot of thanks for the reply! Will implement what you just showed to me first and will come back to the last questions with simplification again.

plamen22:11:26

ok, the answers added a lot of light to my questions 2,3 and 4, but I still struggle with 1. I simplified it to:

plamen22:11:37

how to transform the map of maps

plamen22:11:43

{:name “root” :a {:name “a” :i 1 :j {:name “j” :m :test} :k 2} 😛 {:name “b” :m {:name “m” :n :test}}}

plamen22:11:21

(instead of a smiley, a keyword 😛 should be in the text…)

plamen22:11:39

(instead of a smiley, a keyword “:b” should be in the text…)

plamen22:11:49

to the map

plamen22:11:55

{:name “root” :package “root” :a {:name “a” :package “root.a” :i 1 :j {:name “j” :package “root.a.j” :m :test} :k 2} 😛 {:name “b” :package “root.b” :m {:name “m” :package “root.b.m” :n :test}}}

plamen22:11:04

(instead of a smiley, a keyword “:b” should be in the text…)

plamen22:11:25

sorry for the lack of formatting - learning to use slack

plamen22:11:32

here again: how to trasform

plamen22:11:36

{:name "root"
 :a    {:name "a"
        :i    1
        :j    {:name "j"
               :m    :test}
        :k    2}
 :b    {:name "b"
        :m    {:name "m"
               :n    :test}}}

plamen22:11:50

{:name "root"
 :package "root"
 :a    {:name "a"
        :package "root.a"
        :i    1
        :j    {:name "j"
               :package "root.a.j"
               :m    :test}
        :k    2}
 :b    {:name "b"
        :package "root.b"
        :m    {:name "m"
               :package "root.b.m"
               :n    :test}}}

plamen22:11:55

where the

:package
is constructed recursively from the path defined by the
:name
values

plamen22:11:16

any help is greatly appreciated

nathanmarz22:11:35

@plamen here's one way to do it:

(def NODES-WITH-PATH
  (recursive-path [] p
    [(collect-one :name)
     (continue-then-stay MAP-VALS map? p)
     ]))

(transform [NODES-WITH-PATH :package]
  (fn [& args]
    (let [path (butlast args)]
      (clojure.string/join "." path)
      ))
  data
  )

plamen22:11:14

now starting to understand the benefit of continue-than-stay/stay-then-continue in combination with recursive-path. This was very enlightening (while I discovered Specter 2 days ago and still noob in it, spend may be 3 hours in bending with ALL/FIRST/LAST/p/collect-one for keys/values/recursion and always ended up either missing something from the path or ending in the value of the path key instead of the actual map etc…). A lot of thanks for taking time for my questions! Now everything is explained.