Clojurians Log v2

Thanks for the kind words @nbtheduke!

Is it kibit you are working on?

sort of, yeah

I originally was building an s-expression pattern matching library for use in clj-kondo, but i think it's not quite appropriate given how @borkdude likes to write his code, and then i saw how slow kibit is and in the process of taking over the project, i ended up finishing my own version, which lints much faster lol

gonna see if it's appropriate for usage within kibit itself

cool! looking forward to seeing the fruits of your experiments!

initial report:

$ time lein kibit
...
real    41m41.018s
user    42m18.347s
sys     0m3.742s

$ time clojure -M:run ../netrunner/
...
real    0m14.986s
user    1m47.699s
sys     0m1.404s

Yoiks! That's more than a tad faster!

> but i think it's not quite appropriate given how @borkdude likes to write his code Not sure what you mean by that :P

sorry, i don't mean to insult or denigrate. you've said that you prefer to perform each lint check exactly where it needs to happen as you process the files in an effort to keep speed as fast as possible, instead of pre-processing a given hunk of code and running over it a second time to find errors

that's correct. this is not a matter of preference, but a matter of performance. clj-kondo has to be fast to be a viable option to execute on every keystroke

I'm still thinking about a way to let people have the whole file's rewrite-clj structure and do whatever they want, as a hook. But having kibit as a hook, is probably a little bit on the heavy side, right

Also kibit's matching is a little bit course, since it doesn't really see if a symbol is a local or a var reference, for example, right?

yeah, it doesn't care about that, it's purely focused on shape. if i were to attempt to integrate this into clj-kondo, a bunch of stuff would need to change

it is pretty fast tho. linting the https://github.com/mtgred/netrunner/ with native clj-kondo takes 28 seconds, and linting the same repo with my library as an uberjar takes 10 seconds.

nice. what makes your lib faster than kibit?

macros to generate functions that compare the shape: given (not (empty? coll)), it'll generate something like (and (= :list (:tag form)) (let [children (:children form)] (and (= 1 (count children)) (= 'not (:sym (nth children 0))) ...)

wraps it in a function, and then calling that function is pretty quick

but you could do the same on raw s-expressions right?

does it come down to: not using core.logic is faster than rolling your own matching logic? that's what I would have expected :)

Haha yes, it is due to that

I'm not sure how rewrite-clj makes stuff simpler for you compared to working with s-expressions directly. in clj-kondo I only use rewrite-clj so I have more control over the locations (e.g. numbers can have location metadata) and invalid expressions still parse, etc.

you are aware of grasp right? https://github.com/borkdude/grasp it works directly on s-expressions using clojure.spec expressions

I relied on clj-kondo’s parser because i started this to satisfy my curiosity about alternate methods of writing clj-kondo linters lol

well, clj-kondo still has access to the rewrite-clj nodes so in theory it could give you access to every s-expression it lints to do your custom kibit like matching. Are you aware of the macroexpand hook? It turns a macro rewrite-clj node into an s-expression, which the user can then transform (or lint) and after that it's turned back into a rewrite-clj node.

What you're doing currently is probably more performant though.

I just switched from using clj-kondo's rewrite-clj to the rewrite-clj library itself and running takes longer: 7.6 seconds to 19.2 seconds

i suspect because i had to add a small bit of logic to skip over comment and whitespace nodes

(from (let [~children-form (vec (:children ~new-form))] to (let [~children-form (vec (remove n/whitespace-or-comment? (n/children ~new-form)))]

I've alluded to it before but why aren't you just matching against s-expressions instead of using rewrite-clj?

using edn/read-string?

Oh god no, edn/read-string doesn't work for clojure code. That is very close to what grasp is doing (which I also mentioned before in this thread), if you want to take that approach, you can take a look there.

i don't want to fall down the well of having to update my code to handle new edge cases in clojure parsing, you know? better to rely on the existing libraries with nice apis

kibit does that, and is currently broken if you use :as-alias lol

grasp uses edamame to parse clojure code: https://github.com/borkdude/edamame just look at its source, it's very simple.

ah excellent, i forgot about that library

I mean source as in, look at how grasp is using edamame

i'll read it over

here's my library: https://github.com/NoahTheDuke/spat

warning, i wrote it for myself lol

I'll take a look when I'm not close to going to bed :)

your lib is also rewriting the original source?

I didn't know that was something that kibit did, but I think you could take the same approach with this library: the matching can happen on regular s-expressions like kibit does (which will make it faster than working with rewrite node, I think and the matching logic becomes simpler) and the rewriting can be done using rewrite-clj

I’m only rewriting in the printing to out, not in the file itself

Interesting, I’ll try it on edamame output, see how that feels. The primary logic doesn’t change much thankfully, the output of the macro barely relies on the rewrite-clj form