Fork me on GitHub
#graalvm
<
2022-02-17
>
wombawomba07:02:33

So I'm trying to stop using --initialize-at-build-time without arguments, but the number of classes I have to list to get things to compile is a bit silly (even with graal-build-time). The major reason for this seems to be that sometimes type hints (especially for function return values?) seem to cause a ton of classes to be initialized. For instance, this seemingly innocent type hint forces me to list over a dozen classes: https://github.com/toyokumo/tarayo/blob/master/src/tarayo/mail/mime/multipart.clj#L13 Why do some type hints cause this problem while others seem to be fine? Is there a "best practice" for what to type-hint and what not to type-hint, to make native-image happy? And finally, is there a way to build libraries like this one without having to list dozens of classes under --initialize-at-build-time?

wombawomba09:02:53

Also, how can I use a library to generate data at build time without including that library in the native-image output?

borkdude10:02:11

@wombawomba You can just use a java package prefix in --initialize-at-build-time

borkdude10:02:07

I'm surprised to see type hints cause classes to be initialized at build time. Are you sure that is causing it?

borkdude10:02:03

It's usually stuff like this that causes the problems: https://github.com/toyokumo/tarayo/blob/f9b10b85b7bc1a188d808c3955e258916cd0b38a/src/tarayo/mail/mime/multipart/body.clj#L18 Classes that are initialized at the top level. This can be mitigated by using a delay.

borkdude10:02:07

But if you are sure it doesn't hurt to initialize at built time, just include the package prefix like --initialize-at-build-time=jakarta

borkdude10:02:49

You can even do --initialize-at-build-time=. but this is kind of opposing what the GraalVM authors want you to do

borkdude10:02:06

and you should certainly not do that in an exported native image config

wombawomba10:02:18

Yeah, I've run --trace-class-initialization to confirm that the initialization stems from the type hints.

wombawomba10:02:19

I think jakarta might be fine to initialize fully at build time, but for some of the libs that I'm using I can't initialize do that (e.g. because they initialize a static SecureRandom instance somewhere), so it seems I'm stuck with manually listing classes

wombawomba10:02:48

For cases where both build-time and run-time initialization work, I'm a bit unsure which to pick. What are the pros/cons of each? (I'm not exporting anything.)

borkdude10:02:42

GraalVM tries to encourage to initialize at run time since that is the same semantic you get in the JVM. But with Clojure this is not possible since a lot of stuff happens in static initializers: e.g. resolving things on the classpath. And this stuff cannot happen once the image is already built. Hence we are living in the situation that all Clojure-produced classes must be initialized at build time and also the classes they transitively initialize at build time.

wombawomba10:02:14

Right, yeah, I'm aware of that. I'm curious though how initializing Java packages at build or run time will affect my build size/startup time/performance.

wombawomba10:02:45

..or perhaps it doesn't really affect those things?

borkdude10:02:23

it could incur a tiny bit of startup time when initializing at run time, but other than that, no difference I think

borkdude10:02:31

probably nothing to worry about

wombawomba11:02:45

Alright yeah, I ran some experiments and it seems like it doesn't really matter.

wombawomba13:02:29

Any pointers on how to reduce native-image build size?

borkdude15:02:55

@wombawomba: • use the least amount of dependencies possible • don't use dynamic requires/requiring-resolve/resolve: only use those at the top level

borkdude15:02:32

• avoid loading clojure.spec.alpha specs

borkdude15:02:53

• Use direct linking at compile time (for some reason this helps)

borkdude15:02:03

• Don't use clojure.pprint (babashka contains a patch for it which reduces native image significantly)

wombawomba18:02:47

@borkdude thanks, that helped 🙂

wombawomba18:02:16

it seems like graalvm 22 and static linking with musl helped a bit too

wombawomba18:02:47

I was able to shave off ~200 mb from the final image

wombawomba18:02:11

now I just need to figure out why the image heap size is ~130 mb when I'm only packing 20 mb of resources...

wombawomba18:02:09

phew, okay, https://upx.github.io/ helped me reduce the size by another 75%

borkdude18:02:09

note that upx will reduce size, but will also add to startup time - may or may not matter

borkdude19:02:33

if you are using docker, then that already uses compression so you might not need upx in that case

wombawomba21:02:47

yeah there's a slight increase in startup time, but it isn't a problem

borkdude21:02:20

@wombawomba Great progress. What are you making , if you want to share?