Fork me on GitHub
#beginners
<
2023-02-09
>
zach01:02:22

Hello! I was wondering if folks have any intuitive patterns within clojure for when they find themselves running consecutive maps on data? For example, I am trying to get some deeply nested data, from an object something like this:

[[:a [:foo "y" :matches [[:bar "x" :matches [:baz "b" :line "this is what i want"]]]]
 [:b [...same structure]]
what I’d like to do is get a list of all the line values across this list (in practice it’s thousands of entries in this array) My first pass came from the repl, where I just continually added maps to get more precise to the value I wanted, and so it ended up looking like this:
(def image-lines (apply concat (map #(map :Line (flatten %)) (map #(map :Matches %) (map #(:Matches (second %)) (:Results hound))))))
Which feels messy, but I was just sorta curious if there’s moments when y’all are working in the repl and get into forms like this and can immediately see it needs a different pattern?

phronmophobic01:02:11

It's kind of hard to tell what you're trying to do. The data format from your example looks really awkward. I might try to normalize it or even put it in a database. Having said that, I often use a library called https://grishaev.me/en/zippo/ for generic search. You can use it something like:

(require
 '[clojure.zip :as z]
 '[zippo.core :as zippo])

(def data
  [[:a [:foo "y" :matches [[:bar "x" :matches [:baz "b" :line "this is what i want"]]]]]
   [:b [:foo "y" :matches [[:bar "x" :matches [:baz "b" :line "this is what i want"]]]]]])

(->> (zippo/loc-find-all
      (z/vector-zip data)
      ;; find all the :line keywords
      (zippo/->loc-pred #(= :line %)))
     ;; navigate to the next element in the structure
     (map (fn [zv]
            (z/next zv)))
     ;; unwrap the data
     (map z/node))
;; ("this is what i want" "this is what i want")

zach02:02:04

this is helpful, thank you!

hiredman02:02:40

Nested maps, calling concat, that is all for

skylize18:02:26

This just general info. I have not delved into your example at all to see if fit fits your specific case: If you ignore the automatic coercion of Seqable -> Seq (and of course follow the practice of not injecting side-effects), then map holds the property of being associative. This means that, in general, you can always replace mapping fn f over the result of mapping g with instead just mapping the composition of f and g . All the following should return the same output, with the number of traversals required being the only meaningful difference.

; 3 traversals
(map f (map g (map h m)))

; 3 traversals
(->> m (map h) (map g) (map f)

; 2 traverals
(map f (map (comp g h) m))

; 2 traversals
(map (comp f g) (map h m))

; 1 traversal
(map (comp f g h) m)

; 1 traversal
(map #(-> % h g f) m)

hiredman18:02:55

Map is also lazy

hiredman18:02:42

So traversal accounting is not an open and shut case

hiredman18:02:38

So the 3 traversal case may result in more allocations, but is still O(count(m)) not 0(count(m)*3)

skylize18:02:15

fair points

skylize18:02:08

Regardless, upon full realization, there should be no difference in the final results of any above. 😉

pppaul21:02:36

clojure.walk is helpful for when you know the shape of the data fragment you care about working on, but don't know/care where it is in your data

Nick13:02:58

I would also look into: https://github.com/redplanetlabs/specter . There are some great youtube videos on it showing how it works in practice.

Nick13:02:40

And if you are looking to reshape your data so that you can put it in a database or use a dataset (from http://Tech.Ml) this is a great library as well: https://github.com/turtlegrammar/faconne

Kurosu Chan (Crosstyan)03:02:01

Is there idiomatic way to handle blocking deref in go block? I have some async API requests that return as promise and deref it directly in go block would block the thread pool. Should I change the promise to promise-chan ?

hiredman03:02:20

Yes, or just use a regular channel, depending

Jan K12:02:25

Same as with any other blocking op – wrap it in a thread: (go ... (<! (thread @my-promise))) (but if you can use a promise-chan instead that's even better)