Fork me on GitHub
#clojure
<
2022-07-18
>
Bart Kleijngeld08:07:20

I'm looking for tools/libraries to visualize my code architecture. The idea so far is that: • I want to express the overall structure of the project (namespaces and public functions for instance) in a diagram. • I prefer to write code to get the job done instead of meticulous drawing by hand Most tools I find are UML based, which doesn't seem particularly suited. I have noticed Mermaid.js as an interesting modern candidate. Does anyone have experience with this kind of thing (i.e. modeling FP/Clojure code in a diagram) and can share ideas/tools?

👀 1
p-himik08:07:59

I've used PlantUML before, albeit not for Clojure. Loved it. Seems relatively similar to Mermaid.

Ed08:07:02

I'm the past I've had some success writing something that traversed the code looking for things like custom metadata and output some graphviz dot files (https://graphviz.org/doc/info/lang.html). It depends on what sort of things you want to draw I guess ;).

teodorlu08:07:04

I think you might be interested in the same data as static analysis tools like #lsp and #clj-kondo use. That's also channels I expect you can get good responses to this question.

gratitude 1
p-himik08:07:02

Ah, if you want to visualize the structure of an already existing code, there are plenty of options. Let me know, I'll dig up the links I have saved.

Bart Kleijngeld08:07:46

That metadata approach sounds interesting @U01AKQSDJUX. I was also thinking maybe some libraries exist already which utilize specs (maybe in addition).

Bart Kleijngeld08:07:10

@U2FRKM4TW I would definitely be interested in those links if you don't mind looking them up!

p-himik08:07:01

clograms https://github.com/jpmonettas/clograms > Clojure[Script] source code diagrams > explore and document any Clojure or ClojureScript project by drawing diagrams It's a full-on local web app with lots of features. codegraph https://gitlab.com/200ok/codegraph > Generates a dot file based on Clojure/ClojureScript code. clj-usage-graph https://github.com/gfredericks/clj-usage-graph > A Clojure library that emits usage diagrams for your project. lein-hiera https://github.com/greglook/lein-hiera > Generates a dependency hierarchy graph for Leiningen projects lein-topology https://github.com/testedminds/lein-topology > A Leiningen plugin that generates the data for a Clojure project's function dependency structure matrix. lein-ns-dep-graph https://github.com/hilverd/lein-ns-dep-graph > a Leiningen plugin to show the namespace dependencies of Clojure project sources as a graph. Vizns https://github.com/SevereOverfl0w/vizns > Visualize the relationships between your namespaces and dependencies clojure-dependency-grapher https://github.com/hiredman/clojure-dependency-grapher > script reads the ns forms from clojure files in a directory and writes out a graph of dependences Similar to the above, but works only on deps. Snowball https://github.com/phronmophobic/snowball > View the sizes of your dependencies Generate & display class hierarchy diagrams for Java classes https://github.com/stuartsierra/class-diagram Or just the dependencies https://github.com/walmartlabs/vizdeps Generate dependency graphs for variables in Clojure(Script) namespaces https://github.com/benedekfazekas/morpheus Useful for building own tools https://github.com/clj-kondo/clj-kondo/blob/master/analysis/README.md https://github.com/clojure/tools.analyzer https://github.com/clojure/tools.deps.graph

❤️ 12
Bart Kleijngeld08:07:00

Nice! Thanks for that. I think I came across Clograms btw which sounds perfect for the job. Don't know why I lost track of it.

jpmonettas16:07:46

just for extending the list of Useful for building own tools I would add https://github.com/jpmonettas/clindex, which indexes a Clojure[Script] code base in a datascript db. Clograms is built on top of it.

🙌 1
👀 1
Bart Kleijngeld16:07:18

That looks very useful and interesting. Thanks for sharing!

sreekanthb06:07:19

Any luck with Clograms, it’s not working for me as intended, i tried it on my projects and other famous clj libs..

jpmonettas11:07:39

what problems did you find @U03NT03HAAH? I haven't been using it for some time, but can take a look (author here)

jpmonettas20:07:30

@U03NT03HAAH @U03BYUDJLJF just released com.github.jpmonettas/clograms 0.1.139 with a critical bug fix that was making much of the functionality fail facepalm , just in case you were needing something like it

❤️ 2
sreekanthb10:07:14

Awesome 👍 Will give a try..

Søren Sjørup12:07:49

The parse-* links here https://clojure.org/api/cheatsheet all give 404. Is that just because the articles on have not yet been created, or something technical?

Martin Půda13:07:50

There is already https://github.com/zk/clojuredocs/issues/238 with this problem- ClojureDocs are for Clojure 1.10.1.

Søren Sjørup13:07:27

Thank you for that pointer! 🙏

nixin7219:07:55

So, I’m trying to understand a little bit about how clojure.core.match/match works because I recently got a FileNameTooLongException or something like that from having a huge match. Now, my understanding was that match generates a bunch of lambda functions that of course each generate a class file, and the name mangling scheme produces successively longer and longer names. That was an explanation that I read online, however, when going to investigate this, I can’t figure out what they were talking about. Looking at the expansion of match, there’s no calls to fn* , only primitives like let*, try, do, etc. So where are the lambdas coming from? In the following code, I have a super simple match:

(defn testing [x]
  (match x
    [a] a
    [a _] a
    [a _ _] a)))
Looking at the .class files generated, there’s a file$testing.class for the actual var, file$testing$fn__277.class for the function bound to the testing var, and file$testing$fn__277$fn__278.class which is a last function, but it doesn’t show up anywhere in the macroexpansion… What actually generates this last class file here?

Alex Miller (Clojure team)20:07:53

just fyi, this is a known issue although I don't think anyone has worked on it from either the core.match macro expansion or clojure class naming side

hiredman20:07:35

for certain kind of expressions the clojure compiler transforms them from e to ((fn [] e)) which match may run into as well

Alex Miller (Clojure team)20:07:54

https://clojure.atlassian.net/browse/CLJ-1852 is probably the simplest ticket related to this on CLJ side (but I think there are others)

nixin7220:07:33

Sorry, to clarify, I’ve solved the issue by just rewriting this as a case with some conds. I discovered that this was the issue when I found that old ticket actually. I’m not trying to find a solution to the problem, I want to know what in the compiler is actually generating these extra classes. I know that fn*s generate classes, but there’s no fn* calls in the macroexpansion of match, so where are they coming from?

nixin7220:07:46

In the example above, 3 classes are generated - 1 presumably for the testing var, one for the actual lambda bound to that var, and that leaves a third class unaccounted for by what I know about the compiler. Where is it coming from?

hiredman20:07:39

for complex control flow used as an expression (like let binding the result of a loop or a try/catch) the compiler will hoist the loop or try/catch into a no argument immediately invoked thunk

hiredman20:07:35

those things don't immediately have expression semantics in bytecode, turning them into thunks like that is most immediate way to give them expression semantics

hiredman20:07:40

there is at least one jira ticket about it, but it comes at it from the angle of preserving type hints for the return value of nested loops

hiredman20:07:27

used to be my favorite jira, but I am having trouble finding it

nixin7220:07:46

Hmm… So in the example above there are no try/`catch` or loops that have their output bound in a let* - do you have any other examples of things that might get wrapped in thunks? There’s nothing particularly complex in the expansion here.

hiredman20:07:47

can you share the expansion?

nixin7220:07:12

Woah, 2011 lol

nixin7220:07:44

Yea, here’s the full thing:

(def testing
 (fn*
   ([x]
     (try
       (if (let*
             [and__5531__auto__ (vector? x)]
             (if and__5531__auto__ (== (count x) 1) and__5531__auto__))
         (let* [a (nth x 0)] a)
         (if :else (throw clojure.core.match/backtrack) nil))
       (catch
         Exception
         e__8036__auto__
         (if (identical? e__8036__auto__ clojure.core.match/backtrack)
           (do
             (try
               (if (let*
                     [and__5531__auto__ (vector? x)]
                     (if and__5531__auto__
                       (== (count x) 2)
                       and__5531__auto__))
                 (let* [a (nth x 0)] a)
                 (if :else (throw clojure.core.match/backtrack) nil))
               (catch
                 Exception
                 e__8036__auto__
                 (if (identical?
                       e__8036__auto__
                       clojure.core.match/backtrack)
                   (do
                     (try
                       (if (let*
                             [and__5531__auto__ (vector? x)]
                             (if and__5531__auto__
                               (== (count x) 3)
                               and__5531__auto__))
                         (let* [a (nth x 0)] a)
                         (if :else
                           (throw clojure.core.match/backtrack)
                           nil))
                       (catch
                         Exception
                         e__8036__auto__
                         (if (identical?
                               e__8036__auto__
                               clojure.core.match/backtrack)
                           (do
                             (throw
                               (new
                                 java.lang.IllegalArgumentException
                                 (str "No matching clause: " x))))
                           (throw e__8036__auto__)))))
                   (throw e__8036__auto__)))))
           (throw e__8036__auto__)))))))

hiredman20:07:40

the trys inside the catch might get hoisted, as they might when not the final expressions in the dos

hiredman20:07:33

I don't really recall though, and making what the jvm bytecode provides for try/catch behave like an expression is tricky

hiredman20:07:15

the if is likely hoisted

hiredman20:07:42

ifs also get hoisted if not in the return context I think

nixin7220:07:10

Okay, so basically things that generally map to statements in Java could potentially get hoisted because there’s no expression equivalent?

hiredman20:07:31

clj-701 and this core.async issue I've been working on recently actually both hinge on know what variables are free in an expression, the clojure compiler doesn't tell you that, and for the core.async issue I had to add a tools.analyzer pass to figure it out

hiredman20:07:42

no, sort of the inverse

hiredman20:07:03

oh wait, yes, I think I just misread you

hiredman20:07:41

yeah, things that are statements in java might get hoisted into a thunk to turn them into expressions when used as expressions in clojure

nixin7220:07:01

Right, yea, that’s what I was trying to say

nixin7220:07:48

Hmm, okay, that makes a lot of sense. So here though all of the ifs are getting used as expressions really except the last one - the last one both branches throw. Why aren’t all of them except that one then getting hoisted?

hiredman20:07:30

I think it is only

(let*
                     [and__5531__auto__ (vector? x)]
                     (if and__5531__auto__
                       (== (count x) 2)
                       and__5531__auto__))
where it is hoisted, because that is what the compiler calls an expression context, I think the others are all either a statement context or a return context, and maybe they don't need hoisting in those texts

hiredman20:07:57

huh, actually that code is duplicated in two places, so I would expect a thunk each

nixin7220:07:16

Okay, so something like

(do 
  (if true "a" "b")
  true)
This would be a statement context, because the output of the if isn’t used to actually do anything? Like a statement would be. Then something like
(defn f []
  (if true "a" "b"))
Would be a return context since it can immediately be returned from the function, but something like
(let [x (if true "a" "b")]
  x)
Would be an expression context because there’s no way that it can just immediately return from the function? So here the if would need to get hoisted into a thunk?

nixin7220:07:57

And both of these cases that you pointed out are an expression context because the output of the if is used as the condition for another if, so since it can’t just immediately return from that expression, it needs to wrap it in a thunk to use it in an expression context? And both of these cases that you pointed out, the

nixin7220:07:43

Okay, this makes a lot of sense. But then why is only 1 class generated, and not two? Is that maybe an optimization since both classes would end up basically identical? Besides the constant being used to index the vector?

hiredman20:07:43

it would be an optimization, but I very much doubt the compiler does that, I don't have the compiler open at the moment, so you'll have to crack it open if you want to do anymore digging

nixin7221:07:03

Hmm… Well, I don’t even know where I’d start looking into the compiler to discover something like this, so I suppose I’ll have to be satisfied with this answer for now hahah 😅 maybe I’ll dig into it in the future. Thank you so so much for bearing with me and answering all my questions, this was super interesting an informative!