This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-04-03
Channels
- # announcements (5)
- # aws (3)
- # babashka (52)
- # babashka-sci-dev (23)
- # beginners (51)
- # calva (191)
- # clj-commons (18)
- # clj-kondo (11)
- # cljdoc (39)
- # cljsrn (3)
- # clojure (24)
- # clojure-czech (3)
- # clojure-dev (2)
- # clojure-europe (15)
- # clojuredesign-podcast (2)
- # clojurescript (8)
- # conjure (2)
- # core-typed (151)
- # cursive (15)
- # data-science (3)
- # datalevin (4)
- # datomic (8)
- # figwheel-main (21)
- # fulcro (37)
- # gratitude (3)
- # honeysql (1)
- # hyperfiddle (2)
- # introduce-yourself (1)
- # malli (3)
- # membrane (54)
- # off-topic (21)
- # other-languages (4)
- # portal (18)
- # re-frame (12)
- # reagent (7)
- # releases (2)
- # sci (64)
- # spacemacs (14)
- # sql (2)
- # vim (4)
- # xtdb (6)
BTW I had an idea on how to support inference via cljs.core/type
in the CLJS checker. How important is this function in CLJS? and are namespaced keywords usually used to identify a type?
More noob noise for you if you're interested:
1. Are you suggesting to call cljs.c/type
from inside core.typed
on forms statically? Doesn't that only work "at runtime", as that's when the object whose constructor you're interested in exists?
What I'm speculating about is how to type check something like this:
(def my-specially-tagged-val ^{:type ::my-specially-tagged-val} {:foo 1 :bar 2})
(defn process-thing [thing :- t/Any]
(case (type thing)
::my-specially-tagged-val (+ (:foo thing) (:bar thing))))
Idea: Have a registry that maps :type
values to their types.
(t/register-metadata-type ::my-specially-tagged-val '{:foo t/Int, :bar t/Int})
And once we learn, for example (= ::my-specially-tagged-val (type thing))
, we update the type environment with thing :- '{:foo t/Int, :bar t/Int}
.
Typed Clojure uses an environment of propositions that contain knowledge of the current lexical scope. You can see propositions encoded in the types of predicate, eg., pos-int?
learns the 0th arg is an int if it returns true:
https://github.com/typedclojure/typedclojure/blob/main/typed/lib.clojure/src/typed/ann/clojure.cljc#L2037-L2044

You can extend this propositional logic to reason about "paths" into local variables. Currently, we can reason about the "Class" of a variable, see the type of clojure.core/class
. https://github.com/typedclojure/typedclojure/blob/main/typed/lib.clojure/src/typed/ann/clojure.cljc#L1760-L1762
That enables inferring cool stuff like this:
(defn foo [a :- t/Any]
(when (= Number (class a))
(inc a)))
Now the idea here is to add a Type
path element, so we can reason about clojure.core/type
in the same way.
Also thinking about how Typed Clojure might integrate with clj-kondo (JVM). If the checker can be hooked in every time clj-kondo thinks an expression needs linting, I think that should work.
that's probably the first step to thinking about Graalvm support--integrate via JVM first.
Forgive my ignorance, what is it about graal that makes it so that it needs special consideration for core-typed? What does core-typed do that isn't "regular clojure 'user-space'-y code"? btw I'm very invested in core-typed (or until, as I joke, I stop working at a clj startup lol) in a personal agenda sense but also curious about it - and I'd love to contribute in some way or other someday, if that's something you're into
Good question, the problem is that clojure on graalvm cannot perform macroexpansion.
This (being able to macroexpand forms) was very fundamental to how Typed Clojure worked in its original design. In the last few years, we've added custom rules for particular macros.
@U037TPXKBGS in terms of contributing, it might be fun and informative to try and type check one of your favourite libraries or programs (~500 LOC). As you go through, through it, leave yourself a bunch of comments about what sucks. Then you have a list of relevant features to add to Typed Clojure! That's exactly what I'm doing now with malli.
if you pick a popular library, we can take the annotations and rules you generate from the exercise and make that itself a contribution to Typed Clojure.
1. š® I did not know that about graalvm. What's the best resource to learn more about that? I'm new AF to clj in general, but it does seem that macros are a pretty big part of it, no? Curious about how that is developing. I was so floored when I first read about graal like years ago, been out of the loop!
2. I'm very open to your proposal, however, I'm probably the worst person to ask :
a. I started clj programming like 5 months ago after joining my current org
b. And primarily am working so far in the UI stack in cljs, which I understand is quite far from clj as far as progress/integration with most tooling goes, including typed
, right?
c. That being said, I'm very comfortable throwing my manager @U4YGF4NGM under the bus here,
i. as (I like to believe) he wants nothing more than to make me happy (with cljs tooling),
ii. and is far more experienced
1. in the clj/s OSS community
a. and the experiences with/expectations for/frustrations with them that developers have,
2. but also from his own libraries, like #helix,
a. which I'm sure I can irritate him enough to eventually fully type annotate
3. and has also had experience with the gradually-typed static type annotation world of TS
3. Are you familiar with Typescript's DefinitelyTyped repository?
a. If not, it's a community repo with auto build system set up by M$ for people to share hand-written type annotations for popular libraries whose maintainers don't want to or can't provide them themselves.
b. It's an imperfect solution to an imperfect problem, but provides a halfway-decent opt-in way to essentially let people who need them obtain and contribute annotations
Regarding making the annotations and rules generated from your proposed exercise contributions to Type Clojure, do you mean something like (3) above?
1. I don't know a lot about Graalvm except that babashka and clj-kondo use it, and they are both excellent tools. I donate to @borkdude so I don't have to know too much about graavlm xD
2. FYI Typed Clojure is both older than clj-kondo/malli/spec/schema and both less mature and more experimental. It effectively has no users AFAIK. Just to say, it might be a while, if ever, if it can be applied to your job. That said, I think it's extremely rewarding to work on, and it's really making strides lately.
3. Yes, my plan is just to shove annotations into the monorepo with Typed Clojure so I have everything in the same place.
I'm basically proposing that you write annotations like you would in DefinitelyTyped for library "X", except you also call t/check-ns-clj
on the actual library with those annotations (actually, not sure if this is the usual practice with DefinitelyTyped). Now you have annotations that have been self-checked, and some experience under your belt as to the rough edges to Typed Clojure and some idea of which things you think would be the biggest improvements.
something like https://github.com/lilactown/cascade or https://github.com/lilactown/pyramid might be fun to try and type
that being said, I did my time in the OCaml and TypeScript mines and came out a better programmer and not excited about writing more types for my libs
oh boy, cascade would be a good stress test of how first-class the typings of core ops are.
my experience typing similar libraries in ts like ramda makes me believe it's just a hard problem š
Hahaha speaking of typing rambda you guys will fucking go nuts over this: https://www.freecodecamp.org/news/typescript-curry-ramda-types-f747e99744ab/ Loops in the type annotation syntax??? Bananas --- Anyways, I love typing libraries so much, no clue why, so I'm down to type either @U4YGF4NGM, though IDK what Cascade is
Nice, conditional union types has always had me jealous. I almost included something like it for my PhD thesis IIRC because I wanted the expressivity.
I'm currently playing around with a nutty idea to solve how to check (map (fn [x :- InferThisTypeAutomatically] ..) [..])
. I think Flow lang can do this. I don't think TS can, I believe it infers x
as any
.
Looks like TS added support for this https://www.typescriptlang.org/docs/handbook/2/functions.html#inference
apparently called contextual typing https://www.typescriptlang.org/docs/handbook/type-inference.html#contextual-typing
An old spec detailing when contextual typing happens https://github.com/microsoft/TypeScript/blob/main/doc/spec-ARCHIVED.md#423-contextually-typed-expressions
In a typed function call, argument expressions are contextually typed by their corresponding parameter types.
? Seems vague about polymorphic function calls AFAICT.
Unrelated to the polymorphic ts contextual typing stuff, I'm sure you've seen https://github.com/borkdude/speculative, and I wonder if your spec->types endeavor could help you bootstrap the community types repo thing
Yeah it would be interesting to know how useful raw specs are for type checking. For CRUD-style things, it's probably perfectly adequate.
Unfortunately that's exactly the kind of thing that's usually closed source, so it's harder to experiment.
@ambrosebs welcome to elaborate - I can help with both clj-kondo and graalvm
Kind of, it's starting to look more like clj-kondo now. You can make custom rules for macros. But where clj-kondo gives up if it doesn't recognize the macro, Typed Clojure will expand.
Oh ok, we're talking about the same thing. Typed Clojure has hooks in the same way. But it also expands the macro from its definition if there is no hook.
Yes. My thoughts are, let's try and integrate with clj-kondo JVM first. Try and teach clj-kondo about what Typed Clojure thinks a cache should contain (ie., when to relint a form/file). And hook into clj-kondo's linting at the form-level. ie., if clj-kondo thinks a form needs linting, it also passes it to Typed Clojure.
What would be interesting for clj-kondo is also directly be able to use typed clojure annotations for its own (super simple) type checker
E.g. it can lint the following: (assoc (keyword "dude") 1 2 )
as invalid, but it doesn't know the type of everything
I had exactly the same thought as you, I'm still mulling it over. Ignore my negative reaction for now, thinking out loud xD
@ambrosebs I've got a thing for clojure.spec here (super spartan, but conceptually it works): https://github.com/clj-kondo/inspector You run clojure.spec at runtime, it emits some type info on disk and clj-kondo can start using it for static analysis
Yes, I saw the same thing in malli a few days ago. I was about to build one for Typed Clojure until I realized there's a large difference in expressibility. Typed Clojure types are very expressive, it's not obvious how to translate.
I wish there was one annotation format to rule them all, so then clj-kondo could just support it statically maybe
yes, you have to convert from super expressive to a lossy representation, but the cool thing is that you will get some linting for free, even though it's not perfect
That's what I was thinking too. But look at the type of assoc https://github.com/typedclojure/typedclojure/blob/c7a251333b9d536b5485c0200dd097400b4d0eb8/typed/lib.clojure/src/typed/ann/clojure.cljc#L1245-L1249
it's at the other end of the type spectrum: super mundane, but sometimes it catches a silly thing :)
Type checking with polymorphic types is expensive. If you support polymorphic types in clj-kondo, you'll need to navigate that problem.
So that's why I'm thinking, let's keep clj-kondo's type system. clj-kondo + Typed Clojure is almost another use case, it's going to be very slow.
@borkdude is there a way to hook into clj-kondo's linting on a form-level (JVM, not graalvm)? Say, I just want to add an extra log printing the current form, or call my own linter on the form at the same time as clj-kondo. how might I do that?
@ambrosebs There is no generic hook for that, but for experimentation, we could add that in a branch and then see if that's useful
clj-kondo lints on a file-by-file basis, so you would get every form from file. clj-kondo walks from in recursive descend fashion
clj-kondo runs with a fork of rewrite-clj: the nodes are the same, but all the whitespace has already been erased
Oh, is this pre or post reader conditional expansion? I might also need the current platform. Or, just separate hooks for clj/cljs.
OK, try commit 7c6a60fd1b0c3b168e5982afedbfa41c16b90519
:
user=> (with-in-str "(+ 1 2 3)" (clj-kondo/run! {:lint ["-"] :core-typed-hook prn}))
{:node <list: (+ 1 2 3)>, :lang :clj}
{:node <token: 1>, :lang :clj}
{:node <token: 2>, :lang :clj}
{:node <token: 3>, :lang :clj}
The node->sexpr function is called rewrite-clj.node/sexpr
or you can use the internal version: clj-kondo.impl.rewrite-clj.node/sexpr
ah, now I have a potential list of type errors I want to pass back to clj-kondo to report on.
(at this phase, when this goes public, we need to make everything impl a bit more neat)
Commit:
fe622fe37ff93c2edfdc22c30c4ac65bdcb9ef20
There is a :reg-finding!
key that contains a function that should be called with a map with keys:
row col end-row end-col type message
The location keys are on the metadata of the node. The :type
should be configured in .clj-kondo/config.edn and contain a :level
Could I get an atom that I can store local state in? Stuff like thread bindings for the current ns.
Yes, that's what I was thinking. Either let me set my own state at the top level (say call my hook with zero args and assoc the result to :state
) or give me an atom.
I was thinking a few steps ahead -- I could close over the hook myself (and I'm doing that for the moment), but if the hook was configured in a .edn configuration somewhere, you'd just want to provide a symbol naming the hook I think.
@borkdude do I need to register a new :type
for reg-finding!
to show up in the :findings
entry?
@borkdude how do I get the rewrite-clj sexpr with line/col metadata? Using clj-kondo.impl.rewrite-clj.node/sexpr
seems to give nil metadata.
That's a good question. In general you can't attach metadata to some values like numbers, etc so converting to a sexpr is lossy anyway. But you might be able to postwalk the thing at make a best attempt. Or only report the top level location for now. It's a tricky problem. This is why clj-kondo :analyze-call
hooks operate directly on nodes so you can return more accurate locations to your messages
There's also a :macroexpand
hook which operates on the s-expr but moves the messages to the outer node location
This could maybe be done for the :macroexpand
hook too to get better locations. The challenge there is also that the transformed s-expr is converted back into a rewrite-clj node
Also I think another piece that I can't access is the current file name for *file*
.
PRs welcome against the clj-kondo core-typed-hook branch. I'm done programming for today :)
Just one more thought: if you're more interested in getting the sexprs exactly like tools.reader gives them, maybe it makes more sense to place the hook elsewhere so you can parse the file yourself. I'm not exactly sure where you want to go with this hook.
Yeah I think you're right. Not exactly sure either, my eventual goal is to have something like clj-kondo native but running Typed Clojure. I was hoping this was a good first step, to share as much machinery as I can with clj-kondo. We'll see.
Hehe yes. I looked for a better place for the hook, but I ended up so far up the stack it seemed a little pointless š
Ah, I missed that. This is making me step back to what I think I want to reuse from clj-kondo: ⢠symbol->var resolution, ⢠info about Java classes/methods/fields ⢠participate in existing editor feedback mechanisms
Hmm let's say we wanted feedback from both clj-kondo and typed clojure, but as soon as it's ready. I'm guessing if both were computed by the same run!
then both would have to finish before feedback right?
@borkdude I think the hook needs to be a little higher in the analyzer stack, Typed Clojure just needs the top-level forms, but where the hook is now seems pass the checker all the subforms too.
OK, we can change it to only pass top level forms, or pass a :top-level
key so you can filter