Fork me on GitHub
#polylith
<
2021-08-19
>
seancorfield18:08:38

In case anyone finds this useful: https://gist.github.com/seancorfield/f7aeb115b33f13cc21f8dbbf4f408706 -- it expects to be run in the workspace folder, with tools.deps.alpha on the classpath (hence I run in via clojure -X:poly sean/missing-deps), and it performs some consistency checks between :dev/`:test` aliases and bricks' deps.edn and projects' deps.edn files.

👍 8
❤️ 3
3
Karol Wójcik19:08:00

Could it be merged upstream?

seancorfield19:08:32

I don't know where it would fit in Polylith, nor whether the logic is universally applicable. I'd be interested to hear what folks say about running it on their own projects. It is "just a report" because there are good reasons for not having exact overlapping/disjoint sets of dependencies -- we have Jackson libraries pinned in one of our projects but we don't have it pinned in :dev (at least, not yet) so that is reported under project/dev deps mismatch. It also assumes you're using :local/root and :extra-deps in :dev rather than the (currently) more common :extra-paths to the src and resources folders. The latter is because Cursive doesn't understand sources coming from :local/root deps I believe?

imre20:08:48

Sean, this is a bit off-topic but I noticed (reduce into #{}) in the gist. Is there any specific reason you use that instead of (into #{} cat)? It's the second time I notice this use of into with reduce today. Also, the entire thread this is part of could be run as a transducer inside an into, any reason you aren't using that?

seancorfield20:08:36

No real reason. I built the code up incrementally and just threaded it together. It was pretty much scratch code so I didn't spend any time refactoring it.

2
seancorfield21:08:46

@U08BJGV6E You made me curious so I changed it to a transducer form -- but it isn't quite (into #{} cat) -- because it needs to cat items to get a flat collection and then remove certain things. Here's the old code:

(->> all-files
              (filter #(str/ends-with? (str %) "/deps.edn"))
              (map #(t/slurp-deps %))
              (map #(into (set (when-not (= :test type) (:deps %)))
                          (set (when-not (= :dev  type)
                                 (-> % :aliases :test :extra-deps)))))
              (reduce into #{})
              ;; just the actual libraries:
              (remove #(:local/root (val %)))
              (set))
and the new:
(into #{}
               (comp
                (filter #(str/ends-with? (str %) "/deps.edn"))
                (map #(t/slurp-deps %))
                (mapcat #(into (set (when-not (= :test type) (:deps %)))
                               (set (when-not (= :dev  type)
                                      (-> % :aliases :test :extra-deps)))))
                ;; just the actual libraries:
                (remove #(:local/root (val %))))
               all-files)
The old code probably should just use mapcat there instead of map followed by reduce.

seancorfield21:08:34

There were a few other ->> expressions that could easily become (into #{} xf all-files):

sources   (into #{}
                        (comp
                         (filter #(str/ends-with? (str %) "/src"))
                         ;; we want just the project folders:
                         (map #(str/replace (str %) #"/src$" "")))
                        all-files)
        tests     (into #{}
                        (comp
                         ;; but we do want test folders:
                         (filter #(str/ends-with? (str %) "/test"))
                         (map str))
                        all-files)

seancorfield21:08:34

And this one further down in the gist:

{:missing-dev-deps
     ;; verify all source :local/root deps are present:
     (set/difference
      sources
      (into #{}
            (comp (filter :local/root) (map :local/root) (map #(str root %)))
            (-> root-deps :aliases :dev :extra-deps (vals))))

imre21:08:24

Nice! Using transducers (mostly with into) became a habit for me in the past few years and I was curious if there was a hidden caveat against them here. The first form after your refactor ended up almost what I had in mind (I might have left map and cat different steps)

seancorfield22:08:28

I need to get into the habit of using transducers in my RCFs when I'm gradually building code up. That way I don't have to remind myself to refactor a ->> pipeline into transducers after the fact 🙂

seancorfield18:08:25

(I use this at work to catch missing dependencies/bricks in my aliases, as well as consistency between :dev and "all" of my projects)

seancorfield19:08:32

I don't know where it would fit in Polylith, nor whether the logic is universally applicable. I'd be interested to hear what folks say about running it on their own projects. It is "just a report" because there are good reasons for not having exact overlapping/disjoint sets of dependencies -- we have Jackson libraries pinned in one of our projects but we don't have it pinned in :dev (at least, not yet) so that is reported under project/dev deps mismatch. It also assumes you're using :local/root and :extra-deps in :dev rather than the (currently) more common :extra-paths to the src and resources folders. The latter is because Cursive doesn't understand sources coming from :local/root deps I believe?