Fork me on GitHub
#beginners
<
2021-04-09
>
Célio Cidral01:04:29

Hi all. I’m not sure whether this is the right forum to ask this question (this is going to be a long post) so please feel free to point me out to the right place. I was looking for a function to “extract” values from different places deep inside a map, using something that resembles pattern matching (doesn’t need to be pattern matching though). For example suppose you have this map copied from some deps.edn file:

(def deps {:deps {'acme/spaceship {:mvn/version "1.0.0"}
                  'org.clojure/clojure {:mvn/version "1.10.1"}}
           :aliases {:moon {:extra-deps {'moon/lander {:mvn/version "0.8.7"}
                                         'rock/examiner {:mvn/version "4.0.21"}}
                            :override-deps {'foo/bar {:mvn/version "1.2.3"}}}
                     :mars {:extra-deps {'mars/rover {:mvn/version "1.14.9"}
                                         'comms/serial {:mvn/version "2.4.2"}}
                            :default-deps {'hello/world {:mvn/version "9.8.7"}}}}})
I want a list of all deps, including deps that are deep inside aliases, plus their respective locations (or paths) inside the map. So my first question is: is there such function available somewhere? I couldn’t find such a function, so I made one. Here’s how it works using the example above. I’ll first define a list of “patterns” used to locate the deps:
(def patterns [[:deps]
               [:aliases keyword? #{:extra-deps :override-deps :default-deps}]])
I’ll try to explain how those patterns work but I think you probably already get the idea. Here we have two patterns. Patterns are used to match paths to values inside the map, so if a value exists in a path that matches any of the patterns, the value is returned along with its path. Each element in the pattern can be: • A keyword • A value • Or something that implements IFn If it’s a keyword or something that doesn’t implement IFn, it’s tested for equality. Otherwise it’s called as a predicate function on the map’s keys. Now I call the extract function like this:
(extract patterns deps)
And it returns this list:
({:path [:deps], :val {acme/spaceship {:mvn/version "1.0.0"}, org.clojure/clojure {:mvn/version "1.10.1"}}}
 {:path [:aliases :moon :extra-deps], :val {moon/lander {:mvn/version "0.8.7"}, rock/examiner {:mvn/version "4.0.21"}}}
 {:path [:aliases :moon :override-deps], :val {:foo/bar {:mvn/version "1.2.3"}}}
 {:path [:aliases :mars :extra-deps], :val {mars/rover {:mvn/version "1.14.9"}, comms/serial {:mvn/version "2.4.2"}}}
 {:path [:aliases :mars :default-deps], :val {:hello/world {:mvn/version "9.8.7"}}})
Here’s the implementation:
(defn- key-matches?
  [p k]
  (if (or (keyword? p) (not (ifn? p)))
    (= p k)
    (p k)))

(defn- extract-p
  [pattern val path]
  (let [p (first pattern)
        ks (filter #(key-matches? p %) (keys val))]
    (if (seq pattern)
      (map (fn [k]
             (extract-p (rest pattern)
                        (get val k)
                        (conj path k)))
           ks)
      {:path path :val val})))

(defn extract
  ([patterns m]
   (flatten
    (map (fn [pattern] (extract-p pattern m [])) patterns))))
Now my second question is, do you see any problems with this implementation, or any opportunities for improvements? I think my biggest concern is how it nests calls to map, which is why I use flatten at the end. I wonder if this could result in a significant performance penalty?

Célio Cidral12:04:39

Something like that. Thanks!

delaguardo12:04:59

This library is still under develop but feel free to have a look at implementation

Célio Cidral13:04:41

@U04V4KLKC Very interesting. I’m trying to understand how the registry works (the first argument passed to m/matcher).

delaguardo13:04:54

https://github.com/DotFox/matchete/blob/main/src/dotfox/matchete/compiler.cljc#L287-L292 first argument is a map of identifier -> pattern which you can use later in the pattern given as a second argument for m/matcher wrapped as a [:ref id] sorry for such bloated explanation ) I’m still working on proper documentation for the lib

delaguardo13:04:11

default-registry function places ::custom (atom {}) as a container for every custom patterns you might want to bring in as a first argument for m/matcher. And during compilation every pattern like [:ref id] will use this atom to look up a matcher to use

seancorfield01:04:19

Take a look at Meander and Specter.

👍 1
seancorfield01:04:11

Both of those are designed to help you navigate deeply nested maps.

Célio Cidral01:04:36

Thanks a lot, I’ll take a look!

grazfather02:04:55

are there some examples of just… really exemplary clojure code that makes a good read?

🙏 1
seancorfield02:04:17

I don’t know. A lot of library code has to do “weird” stuff (which is why the library exists), so I suspect most of the “exemplary” Clojure code is in proprietary applications…

seancorfield02:04:49

I certainly wouldn’t say any of my OSS projects are “exemplary” code since they mostly have to jump through hoops to get their jobs done. I’m not sure I’d say our code base at work is “exemplary” either but some parts of it are nice (and some parts are… not so nice) out of 113k lines…

seancorfield02:04:42

(but then it spans a decade of learning Clojure followed by adopting various idioms over time)

blak3mill3r05:04:56

I agree with @seancorfield that it depends on what kind of exemplary you're after, but I will offer an opinion that this library is some of the finest Clojure that I've seen: https://github.com/sicmutils/sicmutils

👍 2
blak3mill3r05:04:31

That does not mean that it is particularly easy to grok, just that it is very well thought out

seancorfield06:04:53

True, there’s a big difference between “well-designed” and “easy-to-read”.

💯 3
grazfather13:04:28

I guess I’ll take a bit of both. Thank you both.

popeye07:04:50

how to fetch the value from Variadic Functionsin clojure

popeye07:04:02

(defn hello [greeting & who]
  (println greeting who))

popeye07:04:14

how to get individual values from who?

Rolfe Power07:04:58

It's a sequence

Rolfe Power07:04:23

(defn hello [greeting & who]
  (println greeting (first who)))

yuhan07:04:51

They're regular sequences, you can use any core function like first , rest, nth , or destructure it further, or map over it etc.

tvirolai07:04:12

You could do something like this: (defn hello [greeting & [firstarg secondarg]] ...)

yuhan07:04:43

eg. (doseq [x who] (println x))

popeye07:04:46

ohh got it thanks

practicalli_john08:04:28

@popeyepwr to return a nicely formatted string from a variadic argument, something like this

(defn hello [greeting & who]
  (str greeting ", " (clojure.string/join ", " who)))

(hello "Welcome" "fred" "betty" "wilma")

hindol08:04:33

I am pretty sure clojure.string/join and str will individually use Java's StringBuilder internally, but we still end up creating an intermediate string via (clojure.string/join ", " who). I wonder if there is one efficient string joining library out there. Something like,

(make-string **somehow represent the complex join logic here**)
(Maybe one should setup everything with interpose et al. first and then call clojure.string/join only once in the end.)

hindol09:04:59

This is excellent! Thanks.

hindol10:04:13

But this is a different kind of optimisation. Not exactly what I described above.

Audrius12:04:29

hi! one question about semantics

(merge {:a 1 :b 2 :c 3}{:a 5 :b 6 :c 7})
=> {:a 5, :b 6, :c 7}
Will it always override first arg's fields with second's ?

tvirolai12:04:28

Yes. For other kinds of merging, merge-with can be used

✔️ 1
🎉 1
zane20:04:11

How idiomatic would it be to accept required arguments via keywords a-la https://clojure.org/news/2021/03/18/apis-serving-people-and-programs?

alexmiller20:04:44

depends who you ask :)

zane20:04:52

I was worried that that might be the response. 😅

zane20:04:34

And if I’m asking you in particular? 🙂

alexmiller20:04:30

even I would say it depends on the situation but generally it's ok by me :)

✔️ 2
borkdude20:04:44

A thought (inspired by Maybe Not I think): what is required now may become optional in the future... and having the required argument positional would make that more difficult (well, you can always pass nil). But making a required argument positional does communicate the intent of required-ness more perhaps.

1
zane20:04:48

> But making a required argument positional does communicate the intent of required-ness more perhaps. Yep! That was why I was hesitating.

sova-soars-the-sora22:04:18

I want to parse an xml file...

(let [input-xml  (slurp "JMdict_e")]
	(println (parse input-xml))
I'm getting a
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by clojure.lang.InjectedInvoker/0x0000000800b8c840 (/.m2/repository/org/clojure/clojure/1.10.0/clojure-1.10.0.jar) to method com.sun.xml.internal.stream.XMLInputFactoryImpl.createXMLStreamReader(java.io.Reader)
So how can I load a file from the project directory ? o.O

sova-soars-the-sora22:04:04

using clojure.data.xml

seancorfield22:04:00

@sova That sounds like you’re using clojure.xml, not clojure.data.xml. The former is “built in” to Clojure itself and is deprecated. The latter is org.clojure/data.xml.

seancorfield22:04:11

(they both have a parse function so a typo in the :require would easily get you the old/bad version instead of the new one)

seancorfield22:04:36

org.clojure/data.xml is solid.

sova-soars-the-sora22:04:02

[clojure.data.xml :refer :all] ought work?

sova-soars-the-sora23:04:38

Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1]
Message: JAXP00010001: The parser has encountered more than "64000" entity expansions in this document; this is the limit imposed by the JDK.
xD

sova-soars-the-sora23:04:58

I switched slurp to .FileReader. and it is working ... 🙂