Fork me on GitHub
#clojure-dev
<
2021-03-07
>
ikitommi17:03:14

1. is there a syntax guide for Var meta :arglists? things like clojure.core/eduction have hand-written syntax '([xform* coll]) 2. are there better ways to extract the arities from normal defns? 3. what ways there are to pull out return type hints per artities, besided Var :tagmeta - which is shared for all arities? 4. what ways there are to get argument type hints? 5. would it be possible to extract all the info (arities, basic type hints) programmatically in the future? e.g. compiler option to inject more info into Var & fn meta? could use the stellar tooling from @borkdude (sci, edamame & clj-kondo), but as the clojure compiler knows all these already, would like to hear if it is possible to get access to that info directly

bronsa17:03:34

every argvec for each arity can specify an arity specific return type hint

bronsa17:03:11

the algo for return type is to first check on the argvec and fallback on the var type hint

bronsa17:03:49

for args it's per binding in the argvec of the specific arity

bronsa17:03:41

the compiler does no validation on 'correct' arglist format, so sometimes handmade ones are not in the right format and they're just ignored

bronsa17:03:17

afair there's no exposed api from the compiler to get this info

bronsa17:03:22

and no specification

bronsa17:03:16

(apologies for being terse, typing from a phone :) )

Alex Miller (Clojure team)17:03:25

there is no formal spec and in the wild you can encounter things that deviate from what you see in core

Alex Miller (Clojure team)17:03:51

in general, "locking" a var into a concrete set of arities immediately raises questions about what that means for future (additive) modifications to the signature

bronsa17:03:45

Well but the compiler does specialise interop calls according to the "current" arglist so it is locking in a way

bronsa17:03:14

ah, you said additive :) fair enough

borkdude18:03:25

> the compiler does no validation on 'correct' arglist format, so sometimes handmade ones are not in the right format and they're just ignored does the compiler do anything with the handmade arglists at all other than adding it to the metadata?

bronsa18:03:17

it doesn't know if it's handmade or installed by defn

bronsa18:03:57

it just looks for type hints if the arglist is present, if the format is incorrect it may ignore it or derive the wrong type hint at callsites

borkdude18:03:06

"it" = the clojure compiler?

borkdude18:03:57

I'm surprised since I assumed that the "handwritten" arglists was just for documentation clarification

borkdude18:03:08

e.g. when you define a function using comp or so

bronsa18:03:22

it serves two purposes, documentation and holds type hints

bronsa18:03:36

there was a proposal years ago about decoupling for this reason

bronsa18:03:14

so you could have a different arglist for documentation that doesn't need to be 'correct', while keeping the true one for the compiler

bronsa18:03:54

but as things are you just need to be careful if you're crafting a tyoe hinted arglist manually

bronsa18:03:11

which to be honest not many do anyway, apart from some instances in core

borkdude18:03:36

if you want to optimize you might as well write it out in full

bronsa18:03:39

so not particularly high value I'd say, even if the idea would be nice

bronsa18:03:04

optimise in what sense?

borkdude18:03:09

using type hints

bronsa18:03:28

sorry, being dense :) not following

borkdude18:03:58

e.g. when I define a function using comp but I want some type hint optimization, I might as well not use comp but write my fn using defn

ikitommi19:03:38

tools.analyzeris just for analyzing source code, right? it’s awesome tool, but could the normal clojure compiler expose more info to the runtime? (via compiler option)

ikitommi19:03:43

(->> (jvm/analyze
       '(fn
          ([] "kikka")
          ([x] (inc x))
          (^Boolean [x y] (+ x y))
          ([x y & zs] (apply + x y zs)))
       (jvm/empty-env))
     :methods
     (mapv (fn [{:keys [arglist body tag params]}]
             {:tag (or (:tag body) tag)
              :arglist arglist
              :args (mapv (fn [arg] (select-keys arg [:form :tag])) params)})))
;[{:tag java.lang.String
;  :arglist []
;  :args []}
; {:tag java.lang.Number
;  :arglist [x]
;  :args [{:form x, :tag java.lang.Object}]}
; {:tag java.lang.Boolean
;  :arglist [x y]
;  :args [{:form x, :tag java.lang.Object}
;         {:form y, :tag java.lang.Object}]}
; {:tag java.lang.Object,
;  :arglist [x y & zs],
;  :args [{:form x, :tag java.lang.Object}
;         {:form y, :tag java.lang.Object}
;         {:form zs, :tag clojure.lang.ISeq}]}]

bronsa19:03:17

yeah sure, I wasn't suggesting using t.a

bronsa19:03:51

just pointing to that function to extract the relevant argvec for an arity, which more or less defines the syntax the compiler understands

andy.fingerhut19:03:58

Bronsa knows best, but my understanding is that the Clojure compiler might reveal some or all of this via JVM objects that it creates, but none of it is documented outside of the Clojure compiler source code (and perhaps someone's personal documentation notes), and none of it is promised not to change across versions of the Clojure compiler

ikitommi19:03:48

this kinda thing:

(let [f (fn
          ([] "kikka")
          ([x] (inc x))
          (^Boolean [x y] (+ x y))
          ([x y & zs] (apply + x y zs)))]
  (pull-out-arities-and-types f))

bronsa19:03:07

no, the clojure compiler only uses arglists during a 'read' and analysis, it's not exposed in the ast

bronsa19:03:27

ah @ikitommi, arglists and type hints are on vars not functions

bronsa19:03:33

there's nothing reified on function objects for that

ikitommi19:03:52

could there be?

bronsa19:03:57

so nothing you can do on lambdas apart from doing reflection over the class

bronsa19:03:15

there could, I believe there's a ticket in jira but no patch

bronsa19:03:12

but also not super useful, type hints are used at callsites, in a HOF setting they would not be analyzeable

bronsa19:03:41

unless doing complex flow analysis and per object specialisation which clojure doesn't do

ikitommi19:03:07

(defn f
  ([] "kikka")
  ([x] (inc x))
  (^Boolean [x y] (+ x y))
  ([x y & zs] (apply + x y zs))) )

(meta #'f)
;{:arglists ([] [x] [x y] [x y & zs]),
; :line 2331,
; :column 1,
; :file ...
; :name f,
; :ns ...}

bronsa19:03:20

that's on the var

ikitommi19:03:59

it has arglists (which is good), but no type-info 😞

bronsa19:03:17

you need to print meta of the symbols and vectors

bronsa19:03:51

if you get the meta of [x y] from that arglists you'll see the boolean tag

ikitommi19:03:27

oh, cool. didn’t know that, thanks!

ikitommi19:03:09

… but the inferring of return types from the body is not there, needs t.a, right?

bronsa19:03:06

clojure doesn't do return type inference, it only does a simple form of local type inference (to specialise interop calls), but doesn't propagate across fn boundaries

bronsa19:03:12

t.a could, and with some minor enhancements so could the clojure compiler, but I believe it's a design decision not to

bronsa19:03:08

what the compiler does is super basic and, I believe, intentionally limited

bronsa19:03:56

remember they are type hints, not type declarations

bronsa19:03:14

there's at least one function in core that has a ^String type hint on an argument that could be a symbol, and to the compiler that's fine

bronsa19:03:58

(since that local is only used on an interop call after a string? check)

ikitommi19:03:29

thanks for the great explanation. Couldn't find the relevant jira issue, only the SO question with few ways to resolve the fn arity at runtime. Would be great if core could expose those directly (in a dev-mode, as extra meta would consume memory)

ikitommi19:03:34

pluggable analyzing would solve both things: t.a could push out arity & infered type hints for both vars and fns.

cfleming20:03:26

TIL about arglists too, I also assumed they were just documentation.