Clojurians Log v2

Hi @jyn514, this is more of a #graalvm question than a clj-easy/graal-build-time question. The Oracle native-image (as opposed to the Community Edition variant) can be asked to produce a build report via -H:+BuildReport. The generated HTML report includes details on the code area and image heap.

I guess I need to freshen https://github.com/clj-easy/graal-docs?tab=readme-ov-file#visualize-what-is-in-your-native-image, I don't think the GraalVM Dashboard is a thing anymore.

Yep, https://www.graalvm.org/latest/reference-manual/native-image/guides/use-graalvm-dashboard/. Here is https://www.graalvm.org/latest/reference-manual/native-image/overview/build-report/.

Oh, and you can use --emit build-report now.

hm, so i did that, but it is not showing me particularly useful information

this is in the "Image Heap" tab

i was hoping it could tell me which classes take up so much space

i guess i could try a binary-search thing where i add and remove dependencies one at a time to see how the heap size compares, but i would rather not 😅 each graal compile takes about a minute

Ah, by packages, you meant dependencies. Hmmm... don't know (or can't remember) a good way to determine how much a particular dependency contributes to heap usage.

hm, ok. i did find this nice chart of code size, but it's mostly java stuff so i don't think it's accurately reflecting heap size

Yeah that's the size of code native-image included. Not heap usage.

It is a pretty report, though, eh?

very pretty, unfortunately not very useful haha

Just like me in my younger days. simple_smile

actually @borkdude you said earlier that you found beholder added a lot of disk size to bb — how did you measure that? just compare the two sizes before and after?

Yep

Just learned about it in this video https://www.youtube.com/watch?v=V_Pls6B4_yg

On macOS:

raspasov@m2-1 jvm25 % clj -J-XX:AOTCacheOutput=cache.aot 
Clojure 1.12.2
user=> Temporary AOTConfiguration recorded: cache.aot.config
Launching child process /Library/Java/JavaVirtualMachines/zulu-25.jdk/Contents/Home/bin/java to assemble AOT cache cache.aot using configuration cache.aot.config
Picked up JAVA_TOOL_OPTIONS: -Djava.class.path=src:/Users/raspasov/.m2/repository/org/clojure/clojure/1.12.2/clojure-1.12.2.jar:/Users/raspasov/.m2/repository/org/clojure/core.specs.alpha/0.4.74/core.specs.alpha-0.4.74.jar:/Users/raspasov/.m2/repository/org/clojure/spec.alpha/0.5.238/spec.alpha-0.5.238.jar -XX:-OmitStackTraceInFastThrow -Dclojure.basis=/Users/raspasov/.clojure/.cpcache/1696584209.basis -XX:AOTCacheOutput=cache.aot -XX:AOTConfiguration=cache.aot.config -XX:AOTMode=create
Reading AOTConfiguration cache.aot.config and writing AOTCache cache.aot
AOTCache creation is complete: cache.aot 26918912 bytes
Removed temporary AOT configuration file cache.aot.config

clj -J-XX:AOTCache=cache.aot 

I used brew install hyperfine to measure

Without cache:

raspasov@m2-1 tmp % hyperfine 'clj'                         
Benchmark 1: clj
  Time (mean ± σ):     472.2 ms ±   8.8 ms    [User: 739.4 ms, System: 58.0 ms]
  Range (min … max):   462.8 ms … 493.1 ms    10 runs

raspasov@m2-1 jvm25 % hyperfine 'clj -J-XX:AOTCache=cache.aot'
Benchmark 1: clj -J-XX:AOTCache=cache.aot
  Time (mean ± σ):     190.7 ms ±   2.3 ms    [User: 382.3 ms, System: 45.8 ms]
  Range (min … max):   187.1 ms … 195.5 ms    15 runs

oh that's quite a speedup, wow

470ms -> 190ms

Are we officially in the “startup times are fast” era? 😜

Would it also make repeated requires across process restarts faster?

Tired of hearing that JVM startup times are “slow”

Video explains details, as far as I understand: yes

i still would not consider 190ms fast tbh, i opened an issue on the Racket issue tracker because i couldn’t get startup times below ~70ms

It actually tunes it specific to your app…

But I just decided to try it with just clj

> i still would not consider 190ms fast tbh, i opened an issue on the Racket issue tracker because i couldn’t get startup times below ~180ms (edited) Haha, well then I guess, “no” 😅 I mean, yeah, if you’re trying to launch a JVM to serve one web request

It’s still a non-starter

The context was a vim plug-in actually

this is still a giant improvement tbc :) it’s just not going to change JVM to the same domain as native binaries

Tbh, I never had a use-case for super fast startup times but I thought it’s interesting, and the speed up is more than 2x, which is cool, with very minimal effort

What do native binaries take? Like 1ms?

raspasov@m2-1 tmp % hyperfine 'python3'
Benchmark 1: python3
  Time (mean ± σ):      16.6 ms ±   0.6 ms    [User: 12.2 ms, System: 3.6 ms]
  Range (min … max):    15.5 ms …  18.8 ms    155 runs

; hyperfine "$(rustup which rustc) --version"
Benchmark 1: /home/jyn/.local/lib/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc --version
  Time (mean ± σ):       5.6 ms ±   0.4 ms    [User: 1.1 ms, System: 4.1 ms]
  Range (min … max):     4.9 ms …   7.4 ms    396 runs

  Warning: Command took less than 5 ms to complete. Results might be inaccurate.

python is not a good benchmark because it does lots of stuff

The JVM apparently, also

And clj obviously adds a bunch to that also

clj -J-Xlog:all=trace   

I killed it after 2 minutes lol

So many logs

lol

now i want to see what rustc does with RUSTC_LOG

$ hyperfine 'bb -e "(+ 1 2 3)"'
Benchmark 1: bb -e "(+ 1 2 3)"
  Time (mean ± σ):       6.1 ms ±   0.3 ms    [User: 2.0 ms, System: 3.1 ms]
  Range (min … max):     5.5 ms …   7.1 ms    312 runs

give me a second to do a dev build

(since you asked for native binary comparison)

I should have thought of that lol! bb

now i want to see what rustc does with RUSTC_LOG

the rust runtime has some stuff that isn't logged but not that much compared to the C runtime, which bb also has

What am I doing wrong here?

$ time clj -J-XX:AOTCache=cache.aot -M -e '(+ 1 2 3)'
[0.003s][error][aot] An error has occurred while processing the AOT cache. Run with -Xlog:aot for details.
[0.003s][error][aot] Loading static archive failed.
[0.003s][error][aot] Unable to map shared spaces
6
clj -J-XX:AOTCache=cache.aot -M -e '(+ 1 2 3)'   0,96s  user 0,14s system 289% cpu 0,380 total
avg shared (code):         0 KB
avg unshared (data/stack): 0 KB
total (sum):               0 KB
max memory:                340352 MB
page faults from disk:     9
other page faults:         23690

try the -Xlog:aot command?

oh yes, I've tried this before. I'm running into this non-empty directory stuff again.

$ clj -J-XX:AOTCacheOutput=cache.aot
Clojure 1.12.1
user=> [1,245s][error][aot] Error: non-empty directory 'src'
[1,245s][error][aot] Error: non-empty directory 'feature-xml'
[1,245s][error][aot] Error: non-empty directory 'feature-yaml'

This is also interesting:

clj -Scp ~/.m2/repository/org/clojure/clojure/1.12.2/clojure-1.12.2.jar   -J-XX:AOTCacheOutput=cache.aot

ah it seemed to record something anyway

time clj -J-XX:AOTCache=cache.aot -M -e '(+ 1 2 3)'
[0.005s][error][aot] An error has occurred while processing the AOT cache. Run with -Xlog:aot for details.
[0.005s][error][aot] Mismatched values for property jdk.module.addmods:  specified during runtime but not during dump time
[0.005s][error][aot] Disabling optimized module handling
6
clj -J-XX:AOTCache=cache.aot -M -e '(+ 1 2 3)'   0,79s  user 0,12s system 350% cpu 0,258 total

Needs Java 25

I'm on 26

$ java -version
java version "26" 2026-03-17
Java(TM) SE Runtime Environment Oracle GraalVM 26-dev+11.1 (build 26+11-jvmci-b01)
Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 26-dev+11.1 (build 26+11-jvmci-b01, mixed mode, sharing)

WOW

haha… hmmm

have not tried 26

I can switch to 25 but I figured I might as well go with the newest graal I can find ;)

I just did clj -J-XX:AOTCacheOutput=cache.aot which worked on JVM 25

I am not sure if the extra stuff makes any difference

Also, macOS, if that makes a difference

ah yes, I'm getting 188ms now

nice, yeah

also on macOS 15.6, m4

M2 Max

macOS 26 beta…

macOS is roughly comparable to linux for process startup (i think the filesystem might marginally slower, but i don't have benchmarks for that). Windows is much much slower on both

macOS used to be a tad slower when it comes to dynamic linking. bb used to have around 20ms startup due to that, but now it's 6. it seems they have improved something

i'm on popOS 22.04, Ryzen 7 7700X 16 hyperthreads

> popOS 22.04 i guess the relevant part is Linux kernel version 6.12.10

raspasov@m2-1 java25 % hyperfine 'java -jar -XX:AOTCache=cache.aot HelloWorld.jar'
Benchmark 1: java -jar -XX:AOTCache=cache.aot HelloWorld.jar
  Time (mean ± σ):      74.0 ms ±   1.4 ms    [User: 19.8 ms, System: 15.0 ms]
  Range (min … max):    72.3 ms …  78.6 ms    38 runs
 
raspasov@m2-1 java25 % 
raspasov@m2-1 java25 % hyperfine 'java -jar HelloWorld.jar'                       
Benchmark 1: java -jar HelloWorld.jar
  Time (mean ± σ):      74.7 ms ±   0.9 ms    [User: 20.8 ms, System: 16.4 ms]
  Range (min … max):    73.4 ms …  76.6 ms    38 runs

Java itself gets no improvement on HelloWorld, which makes sense; this is made for improving startup times on existing apps;

oh interesting, i get different time for bb than borkdude

; hyperfine 'bb -e "(+ 1 2 3)"'
Benchmark 1: bb -e "(+ 1 2 3)"
  Time (mean ± σ):       3.5 ms ±   0.3 ms    [User: 0.7 ms, System: 2.2 ms]
  Range (min … max):     3.0 ms …   4.7 ms    577 runs

so rustc is actually slower than bb by a fair bit

Wonder if that’s something that Clojure can bake in, but I assume the perf improvements are also JVM specific, so probably not.

Whoever needs, it can be trivially done, from Java 25

And I assume layering caches is not a thing, but I don’t know much about it

perhaps this AOT stuff in Clojure also suffers from functions names etc not being deterministic?

> Java itself gets no improvement on HelloWorld, which makes sense; this is made for improving startup times on existing apps; @raspasov think about how AOT caching works: rather than JIT-ing a java program each time it’s executed, it saves the compilation output between runs. HelloWorld has almost no java bytecode, it’s mostly measuring the JVM itself. so there’s nothing to cache.

@jyn514 right, totally makes sense, yes

re: borkdude No idea, what does “not deterministic” mean in this case?

oh because closures get assigned a random name, right

oh…

I see, interesting

(fn [])
=> #object [org.zsxf.query_test$eval44836$fn__44837 0x200be69b "org.zsxf.query_test$eval44836$fn__44837@200be69b"]
(fn [])
=> #object [org.zsxf.query_test$eval44842$fn__44843 0x6d75d50b "org.zsxf.query_test$eval44842$fn__44843@6d75d50b"]
(fn [])
=> #object [org.zsxf.query_test$eval44848$fn__44849 0x2d1c0dd5 "org.zsxf.query_test$eval44848$fn__44849@2d1c0dd5"]

Now I see

i wonder if it’s possible to make that deterministic within a compilation unit my understanding is that the compilation unit in Clojure is one top-level form, so if it’s qualified with the def name i would expect it to be pretty reliable

it would actually be better than rustc if that were the scheme, rustc has all sorts of stuff like spans and dependency versions that affect output

(sorry to keep bringing up rust, it’s just the compiler i know the best lol)

No problem 🙂 Rust working well for you?

in a manner of speaking, i maintained the rust build system for a while

i don’t use it much for personal projects these days

oh wow, ok

Switched to Clojure, or…?

I've done a bit of Rust as a hobby. I remember finding it very hard to keep around a global map of objects due to the lifetime constraints...

clojure for serious things, yeah

played around with weird langs for a couple years, J, Uiua, Rhombus

Rust really really hates global objects, yeah

you can use statics but the language fights you

I feel like that such a rare thing! The stereotypical anecdotal response I hear from Rust people is that Java is “so bad these days” 😄

My problem was: I was building a babashka pod in Rust. When people created a file watcher in bb, I needed to create a filewatcher in Rust and keep track if it in some global state.

I’m… like uhhh… have you tried Java 24, with ZGC ?

i expect most opinions of Java either come from being forced to write Java 8 in university or blog posts about what java was like 20 years ago lol

I mean, I give them a break, such comments in most cases don’t know/make distinction between JVM and Java, so that’s where the problem originates

Yeah exactly

> My problem was: I was building a babashka pod in Rust. When people created a file watcher in bb, I needed to create a filewatcher in Rust and keep track if it in some global state. from this description I don’t understand why you needed the global state, happy to chat about it if you’re interested though

This is the code if you're interested: https://github.com/babashka/pod-babashka-filewatcher I never got around to implementing unwatch which is where I ran into this problem. Mutex<Arc<whatever... I gave up. I made a few comments about it here: https://github.com/babashka/pod-babashka-filewatcher/issues/4

ZGC specifically, appears borderline magic to me!

A true effectively pauseless garbage collector, and in my benchmarks, barely any penalty compared to G1

Perhaps sometimes faster

Arc<Mutex> is roughly equivalent to atom in clojure

When it does GC, the application just slows down a bit, and resumes full speed within a few seconds… so good

@jyn514 switched to golang to implement another watcher. but as we discussed yesterday, filewatchers usually are flaky

that reminds me, i should see if Graal+G1 has a mode for batch processing

i don’t really need GC if the usage is under 150mb or so, my process exits quickly enough

@jyn514 do you mean, no GC until the end of the program? it has such a mode

@borkdude what’s the state of GC in Graal? Is ZGC available there?

(I know very little about Graal)

I actually never bothered to look this up but I don't believe it has ZGC

https://www.graalvm.org/latest/reference-manual/native-image/optimizations-and-performance/MemoryManagement/

@borkdude your last comment there is roughly the approach I would use if I didn’t want to force people to pass in a context, yeah. I think that Mutex::new is const now so you could avoid the lazy-static, but the shape of the solution is right

if I ever want to resume this, I know where to find you :)

the “idiomatic” solution would be to return a Handle type from watch() and have unwatch() take it as an argument

yeah I guess that's the most idiomatic, but I don't know how to do that with RPC style

oh right you have an IPC boundary in the way

you could have the struct handling the RPC calls hold onto a list of watch handles

i don’t know if your pod is multithreaded

(if you’re using the default tokio async executor it is, it’s one of the things i dislike about tokio)

anyway i’ve gotten very off topic haha

gonna go pet some dogs 👋

Here's where I got stuck - generating the aot cache

cd myapp
clj -J-XX:AOTCacheOutput=cache.aot -M:dev:dev-repl:cider-repl
...
nREPL server started on port 54567 on host localhost - 

# connect to the repl and call (System/exit 0)
[67.757s][error  ][aot] Error: non-empty directory 'dev'
[67.757s][error  ][aot] Error: non-empty directory 'test'
[67.757s][error  ][aot] Error: non-empty directory 'src'
[67.758s][error  ][aot] Error: non-empty directory 'resources'
...
Error occurred during CDS dumping
Cannot have non-empty directory in paths

try outside of a project directory

in an empty dir

Sure that works, but I wanted to see how much it could speed up a particular app startup...

I got the same non-empty directory errors when trying it on our app. I wonder if this is just for jars

Just for jars. https://openjdk.org/jeps/483 says "Class paths must contain only JAR files; directories in class paths are not supported because the JVM cannot efficiently check them for consistency."

Those commands as I originally posted need JVM 25

There might be a way to get it running on JVM 24, but might not be as convenient, not sure.