Fork me on GitHub
#core-typed
<
2024-06-24
>
TJ Campanella15:06:19

Is there anyway the process of registering annotations can be sped up? It takes a decent amount of time to the startup process of our application which already had a longish startup time lol.

ambrosebs18:06:52

I assume you're talking about dev-time? I think a lot of time is spent compiling the type checker. Perhaps see if this helps https://clojure.org/guides/dev_startup_time

ambrosebs18:06:32

oh I forgot you're using t/pred which forces all types to collect

ambrosebs18:06:04

so this is affecting your production startup? that's a problem.

ambrosebs18:06:55

do you deploy your app with AOT compiled sources?

TJ Campanella18:06:00

It affects everything really. Whenever we run tests, start the site, type check, certain linters etc. Its not the worst thing but I was just curious if there is anyway to speed that up or remove it entirely. For our production setup it does do the same thing there but we have a rolling update setup so it doesn't cause our service to go down or anything for longer so its honestly more of a dev thing.

TJ Campanella18:06:40

We use lein uberjar to compile the jar

ambrosebs18:06:59

Do you have this in your project.clj?

:profiles {:uberjar {:aot :all}}

ambrosebs18:06:11

that might improve prod startup a little bit

ambrosebs18:06:19

there's a bit more optimization I can do for collecting types

ambrosebs18:06:40

it's quite heavy right now to annotate a type

ambrosebs18:06:35

do you include the checker at production time or just the typed.clj.runtime?

TJ Campanella18:06:50

Yeah that does remove it for the jar thanks.

TJ Campanella18:06:32

We use [org.typedclojure/typed.clj.runtime "1.2.2-SNAPSHOT"], [org.typedclojure/typed.clj.checker "1.2.2-SNAPSHOT"], and [org.typedclojure/typed.lib.clojure "1.2.2-SNAPSHOT"]. I think we experimented previously and we needed those or it wouldn't work since we use t/pred

ambrosebs18:06:15

you should be able to use pred with just typed.clj.runtime and typed.lib.clojure

ambrosebs18:06:48

it's an oversight on my part if not

TJ Campanella18:06:31

Ok let me try that

ambrosebs18:06:58

I will try a few ideas on the loading speedup and get back to you.

ambrosebs18:06:46

removing the checker should have no effect on the loading time but will drastically reduce the # of prod deps

TJ Campanella19:06:45

Sounds good thanks. I was able to move the checker to dev dependencies and everything still works.

👍 1
ambrosebs19:06:52

for production startup, AOT compilation may solve many of your issues. t/pred only loads annotation at macroexpansion time.

TJ Campanella19:06:16

Yeah it cuts the startup time down significantly

TJ Campanella19:06:23

Idk how i never heard of that before

TJ Campanella19:06:48

It pre compiles all of the files that is what we want anyways I wonder why that isn't the default

ambrosebs19:06:36

great. is typed clojure loading anything after aot?

TJ Campanella19:06:53

No it doesn't print out the registering annotations stuff

ambrosebs19:06:02

ok brilliant

ambrosebs19:06:17

Could you send me the list of annotations loaded during dev?

ambrosebs19:06:46

at least the namespaces owned by typed clojure

ambrosebs19:06:03

just need to make sure annotations for the checker's implementation aren't being loaded

TJ Campanella20:06:07

In dev I get

Registering annotations from typed.ann.clojure...
Registering annotations from typed.cljc.runtime.env-utils-annotations...
Registering annotations from typed.clj.checker.assoc-utils...
Registering annotations from typed.clj.checker.parse-unparse...
Registering annotations from typed.clj.checker.subtype...
Registering annotations from typed.cljc.checker.check...
Registering annotations from typed.cljc.checker.check.funapp...
Registering annotations from typed.cljc.checker.cs-gen...
Registering annotations from typed.cljc.checker.filter-ops...
Registering annotations from typed.cljc.checker.filter-rep...
Registering annotations from typed.cljc.checker.lex-env...
Registering annotations from typed.cljc.checker.object-rep...
Registering annotations from typed.cljc.checker.var-env...

ambrosebs20:06:05

hmm ok thanks

ambrosebs03:06:29

latest main now AOT compiles typed.clojure and clojure.core.typed. might give 100ms back during dev.

ambrosebs03:06:57

I'll go through and see what else I can AOT compile.

ambrosebs04:06:45

another improvement, you should only see this log now, all the others won't be loaded:

Registering annotations from typed.ann.clojure...

ambrosebs04:06:55

that should be a drastic improvement on startup since before t/pred would load half the checker in dev.

ambrosebs04:06:54

the tradeoff being the first t/check-ns-clj will take longer. but I'll work on that.

TJ Campanella15:06:00

That’s awesome thanks. It does seem to be much faster now.

👍 1
TJ Campanella20:06:40

I think something in a recent snapshot commit is causing this weird error during our CI pipeline when doing lein uberjar. We are getting this error

Registering annotations from typed.ann.clojure...
Unexpected error macroexpanding tyc/pred at (ourfile.clj:43:12).
Unexpected error (IndexOutOfBoundsException) macroexpanding tyc/pred at (ourfile.clj:43:12).
Method code too large!
That line its referring to looks like (def user? (tyc/pred User)) and User is just a hmap defalias

ambrosebs20:06:11

thanks, reverting

ambrosebs23:06:32

I tried the optimization again, lmk if it broke anything. Compilation should be quicker and probably loading too.

ambrosebs23:06:08

well the build failed, will try later.

ambrosebs23:06:02

there's a flaky test that's worrying me but it's deploying now.