Fork me on GitHub
#clj-kondo
<
2023-04-29
>
Matthew Davidson (kingmob)08:04:46

I'm weighing options for linting the two main macros used to define every operator in primitive-math (variadic-proxy and variadic-predicate-proxy). I like kondo a lot, but it seems like we have many options when it comes to macro linting, and no guide. I could use a flowchart 😉 I originally thought I could use the macros as their own hooks, but then I wondered if :lint-as def or declare would be better, since I mostly just want the various operators to be resolved. Linting as def works, but using def doesn't turn up an error if the operators are used as higher-order fns, which it should, since they're macros. Thoughts?

Matthew Davidson (kingmob)08:04:03

One catch is, they're macro-generating macros.

borkdude09:04:21

Let's start with an example. What often only matters is how the code is used superficially, not what the macros generate in real life

Matthew Davidson (kingmob)09:04:20

Sure, here's varidic-proxy:

(defmacro ^:private variadic-proxy
  "Creates left-associative variadic forms for any operator."
  ([name fn]
     `(variadic-proxy ~name ~fn ~(str "A primitive macro version of `" name "`")))
  ([name fn doc]
     `(variadic-proxy ~name ~fn ~doc identity))
  ([name fn doc single-arg-form]
     (let [x-sym (gensym "x")]
       `(defmacro ~name
          ~doc
          {:arglists '([~'x] [~'x ~'y] [~'x ~'y & ~'rest])}
          ([~x-sym]
             ~((eval single-arg-form) x-sym))
          ([x# y#]
             (list '~fn x# y#))
          ([x# y# ~'& rest#]
             (list* '~name (list '~name x# y#) rest#))))))

Matthew Davidson (kingmob)09:04:23

(I tried dropping it in as a :macroexpand hook, and removeing the ^:private, but it doesn't seem to be working...)

borkdude09:04:25

So you want support for this macro for yourself, not for third party users, am I right?

borkdude09:04:46

(since it's private)

Matthew Davidson (kingmob)09:04:02

Mostly for users, because it defines all the public operators in primitive-math

Matthew Davidson (kingmob)09:04:14

I suppose I could do declare , which would be friendly even to non-kondo users, but I wanted to see how ar I could get with hooks

borkdude09:04:26

ok, then, you can maybe expand in a hook to indeed, (declare +, -, ...)

borkdude09:04:47

or you could even just put declare inside of the code itself, just to satisfy kondo, it doesn't hurt anyone

Matthew Davidson (kingmob)09:04:28

Right, but as I mention in the parent comment, that won't trigger an error if someone attempts to use them as a HOF

borkdude09:04:42

are they macros?

Matthew Davidson (kingmob)09:04:46

(Which is probably rare, but who knows)

borkdude09:04:40

I think clj-kondo currently doesn't warn when a macro is used as a hof, but that might come sometime soon. In that case you could start doing this probably:

(declare ^:macro + ^:macro - ...)

Matthew Davidson (kingmob)09:04:26

Ahh, interesting. And in that case, no hooks are needed, I assume

Matthew Davidson (kingmob)09:04:41

ok, lemme try that. thx!

👍 2
Matthew Davidson (kingmob)09:04:24

Heh, I decided to write a little declare-macro macro, and immediately gave myself the same problem all over again. laughcry

😅 2
borkdude09:04:28

Another way is to get rid of the macrology altogether and just copy paste the +, - etc macros. The low tech solution :)

Matthew Davidson (kingmob)09:04:10

I'm using a Lisp, dammit. I should be able to use macros everywhere!

😂 2
Matthew Davidson (kingmob)09:04:17

More realistically, I'd prefer not to touch the core code of a library as old and widespread as primtive-math. It would probably be fine, but I don't really want to fool with it...

borkdude10:04:48

agreed: > the golden rule is to change nothing - unless we're adding something or fixing a bug (David Nolen, some years ago)

Matthew Davidson (kingmob)10:04:34

Well, this is a fun one:

src/clj_commons/byte_streams/pushback_stream.clj:0:0: error: Can't parse src/clj_commons/byte_streams/pushback_stream.clj, java.lang.NullPointerException                                                                

borkdude10:04:14

I think this has to do with configuration. E.g. this doesn't cause an NPE:

clj-kondo --config-dir /tmp --lint src/byte_streams/pushback_stream.clj

Matthew Davidson (kingmob)10:04:33

Interesting. Maybe some lingering reference in the imported primitive-math config.edn?

borkdude10:04:48

It seems to be related to the :lint-as entries

Matthew Davidson (kingmob)10:04:07

...

{:lint-as {byte-streams.utils/defprotocol+  clojure.core/defprotocol
           byte-streams.utils/deftype+      clojure.core/deftype
           byte-streams.utils/defrecord+    clojure.core/defrecord
           byte-streams.utils/definterface+ clojure.core/definterface
           clj-commons.byte-streams.utils/defprotocol+  clojure.core/defprotocol
           clj-commons.byte-streams.utils/deftype+      clojure.core/deftype
           clj-commons.byte-streams.utils/defrecord+    clojure.core/defrecord
           clj-commons.byte-streams.utils/definterface+ clojure.core/definterface}}

borkdude10:04:24

CLJ_KONDO_DEV=true clj -M:clj-kondo/dev --lint src/byte_streams/pushback_stream.clj
Execution error (NullPointerException) at clj-kondo.impl.namespace/var-classfile (namespace.clj:112).
Cannot invoke "clojure.lang.Named.getName()" because "x" is null

Full report at:
/var/folders/j9/xmjlcym958b1fr0npsp9msvh0000gn/T/clojure-10659668507427392406.edn

Matthew Davidson (kingmob)10:04:24

Seems to be related to the older byte-streams.utils/deftype+

borkdude10:04:06

probably should be some->> instead of ->> or so

borkdude10:04:57

I don't even know what this code does

Matthew Davidson (kingmob)10:04:03

hmmm...what's really wrinkling my brain is the code is identical in the non-depreacated ns, but that's not causing problems...somehow

borkdude10:04:42

ah, it lints if the var names differ only by case

borkdude10:04:41

and somehow a var name is missing there

Matthew Davidson (kingmob)10:04:56

The code between the old and new namespaces is 99% the same

borkdude10:04:57

do the protocol+ macros support the exact same syntax as the original or are there slight differences?

Matthew Davidson (kingmob)10:04:25

Within byte-streams, the deftype+ macros are identical. Will have to check the usages

Matthew Davidson (kingmob)10:04:58

Between projects tho, Zach frequently changed those potemkin macros, so each one has a slightly different functionality, fwiw

borkdude10:04:01

I'll try to debug some more

Matthew Davidson (kingmob)10:04:16

Could it be related to the both macro?

Matthew Davidson (kingmob)10:04:53

(Though that still wouldn't explain why it fails on the old ns, but not the new ns)

borkdude10:04:25

$ CLJ_KONDO_DEV=true clj -M:clj-kondo/dev --lint src/byte_streams/pushback_stream.clj
:ns byte-streams.pushback-stream :name PushbackStream
:ns byte-streams.pushback-stream :name Consumption
:ns byte-streams.pushback-stream :name ->Consumption
:ns byte-streams.pushback-stream :name trigger
:ns byte-streams.pushback-stream :name trigger
:ns byte-streams.pushback-stream :name put
:ns byte-streams.pushback-stream :name put
:ns byte-streams.pushback-stream :name expand-either
:ns byte-streams.pushback-stream :name expand-either
:ns byte-streams.pushback-stream :name walk
:ns byte-streams.pushback-stream :name walk
:ns byte-streams.pushback-stream :name prewalk
:ns byte-streams.pushback-stream :name prewalk
:ns byte-streams.pushback-stream :name both
:ns byte-streams.pushback-stream :name both
:ns byte-streams.pushback-stream :name nil
Execution error (NullPointerException) at clj-kondo.impl.namespace/var-classfile (namespace.clj:113).
Cannot invoke "clojure.lang.Named.getName()" because "x" is null

borkdude10:04:42

maybe it's this?

deftype+ (either [PushbackByteStream] [SynchronizedPushbackByteStream])

borkdude10:04:28

that seems to trigger the NPE

Matthew Davidson (kingmob)10:04:42

Yeah, but the identical code exists in clj-commons.byte-streams, too

borkdude10:04:20

I'm getting the same NPE over there

Matthew Davidson (kingmob)10:04:45

Oh? Let me check that...

borkdude10:04:35

I think we should make clj-kondo more robust, it doesn't handle the absence of a var name properly here, but hopefully you know what to do?

borkdude10:04:24

do you have a hook for both?

Matthew Davidson (kingmob)10:04:37

Not yet. The both/either macros are convoluted, iirc

Matthew Davidson (kingmob)10:04:12

And they're only internal

borkdude10:04:21

well, I think either that, or just ignore the contents of both and then declare what you need to signal the existence of vars to clj-kondo, if they are important?

Matthew Davidson (kingmob)10:04:17

I've considered just separating the classes entirely for a while now. Or adding a property to indicate which is which.

Matthew Davidson (kingmob)10:04:56

Sometimes the fact that Hickey forbade inheritance is super-annoying.

Matthew Davidson (kingmob)10:04:33

Anyway, I know where the problm lays, anyway

Matthew Davidson (kingmob)10:04:43

Thanks so much for looking into this!

👍 2