Fork me on GitHub
#cljs-dev
<
2018-11-22
>
dnolen14:11:09

making good progress on proper multi-arity, variadic fn return type inference

dnolen14:11:40

I think this plus the predicate stuff is probably enough information for helpful nullability checking if you spec an fn

dnolen15:11:11

I looked at letting Closure do this and it would be really ugly - but now that we have so much type info flowing through - might as well use spec to (optionally) tell you when things don't look quite right

mfikes15:11:44

Awesome—the ability to have a distinct return type per arity sounds really good

borkdude15:11:46

spec as in clojure.spec?

mfikes15:11:16

The and / or (and if) inference patch similarly narrows the inferred type of many forms: Instead of a union, you oftentimes get a single type instead of a set. (https://dev.clojure.org/jira/browse/CLJS-2869)

borkdude15:11:45

> helpful nullability checking if you spec an fn I was wondering if this has anything to do with clojure.spec 🙂

borkdude15:11:25

are these macro or really fn specs that are used for this? the more specs you add for core fns, the better the inference?

borkdude15:11:00

anyway, it sounds mind blowing 🙂

dnolen15:11:40

type inference is really just it's own thing

dnolen15:11:03

originally way back when I wanted return type inference so that user fns could get type inference w/o the ugly explicit annotations

dnolen15:11:18

the idea being that we hint the core protocols and fns, and then everybody else gets hinted

dnolen15:11:34

that's what return type inference was originally about

dnolen15:11:53

I didn't want users hinting boolean or other fiddly perf things

dnolen15:11:21

however I had another thought now that the type inference will be pretty accurate

dnolen15:11:38

and return type inference will always give us something meaningful for standard library usage

dnolen15:11:55

even in the higher order case you know that stuff like map never returns nil

dnolen15:11:35

so put those benefits to one side

dnolen15:11:53

a user decides to spec a couple of high level fns

dnolen15:11:03

by default spec is not nullable

dnolen15:11:10

nilable? is explicit

dnolen15:11:28

this means we can trivially compare what you say the fn spec is with what type inference tells us

dnolen15:11:35

bam - nullability checking

dnolen15:11:59

(spec is clojure.spec yes)

borkdude15:11:06

😲 🙇🙏

borkdude15:11:03

if this works out, could it be applied to clojure as well?

dnolen15:11:33

that's up to the Clojure team

borkdude15:11:40

in theory I mean

dnolen15:11:01

also nothing is stopping anyone from doing this as linting thing

dnolen15:11:23

even in the ClojureScript case you probably want to do this when running the whole build, could get fiddly at the REPL

borkdude15:11:39

yeah. I noticed the joker linter already is doing similar stuff

mfikes15:11:26

It seems that type inference alone can sometimes give you info about nullability: If a fn returns string vs. #{string clj-nil}, for example. But yeah, if you see nilable on the spec that's another direct way of deducing that.

dnolen15:11:52

the problem is that type inference doesn't necessarily tell give you intention

dnolen15:11:02

so don't really want to use that as the source of what the user wants

dnolen15:11:24

but the moment a user writes a spec - then they've already taken that step and probably would be happy to be told something is wrong at compile time

mfikes15:11:13

Oh, by "nullability checking", do you mean a subset of type checking, constrained to nullness?

dnolen15:11:32

that's what I'm interested in for the first step

dnolen15:11:39

and we can see if that's delivering value behind a flag

dnolen15:11:45

and then expand if it seems promising

dnolen15:11:24

also stuff like this is actually pretty common in Common Lisp

mfikes15:11:33

There is that other stuff I was doing (not a patch yet, but in a branch), that infers parameter types. From that you can see general type mismatches. But, it is much more aggressive (and difficult) than nullability checking.

dnolen15:11:12

yeah I think inferring parameter types is probably too much - I was thinking about that

dnolen15:11:38

leave that to the user - they write the specs

borkdude15:11:47

and what if the users loads specs to core functions, will this somehow improve things automagically?

borkdude15:11:16

probably not since core fns often accept nilable inputs

mfikes15:11:15

The unsolved problem with parameter type inference (in the branch I was messing around in) is that it gets into situations where you need multiple passes, and you start to feel like you are informally building things that Hindly-Milner solves. If a parameter type can easily be derived from a spec, you more rapidly get to the same place.

dnolen15:11:25

yeah, I think it's just too much work, complicated, slower etc.

dnolen15:11:15

and in the end I actually think not a great fit for Clojure style development

dnolen15:11:57

inferring returns and narrowing w/ predicates gives a lot of info

mfikes15:11:16

Yeah, I’m still worried we may compile code based on inferred types that becomes invalid as types change (REPL case)

dnolen15:11:53

I think we definitely want to put aggressive type driven optimization behind a flag to avoid that

dnolen15:11:56

like :static-fns

mfikes15:11:56

Exactly - :static-fns is about perf related to arity info that can go stale, we may need something for types. (The type info can still exist and be useful for type checks, warnings, etc, but if that new flag were enabled, compiled code would be disallowed from leveraging and statically depending on types for perf reasons)

mfikes15:11:32

Sorry, I realize I’m repeating what you said, just in a different way 😀

mfikes15:11:09

Hrm—maybe the meaning of :static-fns could grow to cover this aspect. (Or that might be confusing.)

mfikes15:11:12

Nah, maybe :static-fns should remain about dispatch, like its more aggressive cousin :fn-invoke-direct

aisamu16:11:30

Hi! def'ing a var in another namespace "works" in 1.10.339, setting the var when the namespace exists or outputting an error when it doesn't. 1.10.439 fails during compilation with an NPE. This is definitely in undefined-behavior land - used as a dev-time-only hack - but I suppose sharing it wouldn't hurt. Here's a minimal repro gist: https://gist.github.com/aisamu/3d860dd329436361113d144a4a162832 (And thank you for the superb work on CLJS! 🙂)

thheller17:11:05

I suspect its related to https://dev.clojure.org/jira/browse/CLJS-712 as well but definitely not intended to work at all

dnolen17:11:40

@aisamu probably not going to spend any time on that one - you can accomplish that with set!

aisamu17:11:34

Yup, that was the workaround! The main issue was finding that as the culprit, since the thrown NPE only points to the file path, with no further cause/location

dnolen17:11:34

I pushed the inference stuff

borkdude17:11:52

def’ing a var in another namespace is not supported in JVM Clojure as far as I know, hence it should probably not be supported in CLJS either

thheller17:11:15

@dnolen any thoughts on the dotted-symbol issue in general? should they even be checked at all or is the unchecked behavior intentional?

thheller17:11:32

the hacky patch I did for shadow-cljs keeps finding small issues like this https://github.com/juxt/bidi/issues/189

thheller17:11:57

don't really hurt since the emitted code is still correct but still kinda incorrect

aisamu17:11:34

Yup, that was the workaround! The main issue was finding that as the culprit, since the thrown NPE only points to the file path, with no further cause/location

dnolen17:11:54

I've pondered but never really bothered as there's too much code out there that uses that pattern due to it being it included since day 1

thheller17:11:20

yeah probably

thheller17:11:26

athough the way it is handled now is definitely incorrect and should be fixed. like the defprotocol thing ending up with cljs/core.-clone symbols

thheller17:11:45

no warning is probably ok