Fork me on GitHub
#fulcro
<
2021-12-21
>
genekim23:12:55

Hello, all — thanks to help from @borkdude, we managed to get a simple Fulcro server running using GraalVM native image. What this means: I was able to get the server deployed into a Docker image and running on Google Cloud Run, and get startup times (as measured by “time to first frame rendered on client”) down from 25-30s in an uberjar, to 8 seconds. Repo that has links to both versions of app here: https://github.com/realgenekim/rss-reader-fulcro-demo Caveats: • all the database adapters were removed (postgres, datomic) for now • minor changes made guardrails to change how async checking via another thread (I’ll post the changes in a forked repo in next week, and will start discussion with @tony.kayon potential PR.) • if I understand correctly, it also required running with newest versions of Clojure and core.async (due to “unbalanced monitor” issue), and running ClojureScript off of github (due to some reason I can’t recall.) Value to me (and hopefully others): • shows viability of an attractive way to Fulcro server in Heroku or Google Cloud Run (currently my favorite way to run apps), without really long cold start times, associated with larger docker images, JVM / Clojure app startup times.

😻 3
👍 4
tony.kay00:12:06

Curious: 1. Guardrails should be disabled in production, and therefore should elide most code. The async stuff should be irrelevant, though I guess it is in the namespace, so GraalVM has to deal with it. Perhaps the async mode needs to go away, and GR needs to do random sampling when specs of a given function are called a lot or are slow. For that matter, on the CLJ side where the async makes any difference at all, it could be implemented without core.async. 2. As far as startup time: why not just use AOT compiled code? The JVM startup time should be pretty fast with AOT'd code. Most startup time is compiling the CLJ files. That has the advantage of not having the complexity of a Graal build, which will be very fragile as you evolve your server, I would think.

genekim00:12:44

1. Not sure I have an opinion on this — I love where you’re going with Guardrails, and I love the idea of asynchronous mode. I do wish that there was some way to, at AOT compile time, elide Guardrails away, or substitute with the “noop” namespace. I’ll have something more concrete to propose in a couple of days. 2. Startup time: the 30 seconds to generate first client frame is with uberjar AOT code. I’ve seen very simple app ring servers start running in about 10 seconds. For reasons I don’t quite understand, it takes considerably longer for Fulcro server to start — I think it’s the overhead of JRE image layers, etc. Here’s the part of the Makefile that runs depstar to generate AOT uberjar, FWIW — let me know If there’s something I did wrong. https://github.com/realgenekim/rss-reader-fulcro-demo/blob/75ffea13e1c798501c6b612d758d88a983ed3d2c/Makefile#L20 Thanks, Tony!

Hukka07:12:08

Ugh, that's interesting and disheartening. We also have thought about using Graal later on, but 8s will not be fast enough. Then it's all the same just to have the JVM running, and make sure it is always running.

borkdude07:12:31

For comparison, the native compiled server starts in a few milliseconds

borkdude07:12:16

You can wrap the thread in a delay so it won't start when you load the code, this should be simple enough.

borkdude07:12:29

But if you can elide guardrails in production by not even loading it, this would be better. This can also be accomplished in user space by just including namespaces that override it and don't do anything.

Jakub Holý (HolyJak)09:12:06

Great job you too! Thank you for exploring this uncharted territory! If "native compiled server starts in a few milliseconds" then why does the Graal-ed Fulcro app take 8s?

borkdude09:12:57

@U0522TWDA Locally it starts in a few milliseconds, but @U6VPZS1EK includes a lot of data on startup that he sends to the client, which goes a lot faster locally. But this is specific to his app and probably can be optimized.

gratitude 1
Jakub Holý (HolyJak)09:12:33

I see, thank you! Now I see what you meant by the delay above.

Jakub Holý (HolyJak)09:12:02

I remember you got some jdbc drivers working with GraalVM so I assume getting at least the SQL adapter of RAD in would be quite feasible,right?

borkdude09:12:49

Yes, pretty much all major SQL databases work

tony.kay16:12:27

I'm grateful that people are trying GVM, and that @borkdude is doing so much to make this more tractable for those that need it.

💯 3
genekim19:12:29

@U8ZQ1J1RR You've got me motivated to try to bring down startup times even more — maybe by using Alpine image instead of Debian. Or even the Scratch image. (The slightly annoying thing for me. I'm using Google Jib, which has amazing Docker image creation times — but it apparently can't set the file executable bit. Which means I need bash or sh to chmod the file.) For me, 8s is fine. It's faster than Heroku, which has worked fine for me for over a decade.)

borkdude19:12:56

For Alpine you need static executables which you can make with musl. I've done this for babashka (its alpine image is 30mb compressed) but it involves some configuration

genekim08:12:51

Wow. I got the time to first frame render to about 2 seconds. I think this is the near lower bounds for Google Cloud Run. The tricks: • -H:+StaticExecutableWithDynamicLibC to link against zlib, so I can use http://gcr.io/distroless/base-debian11 image • use upx -7 -k myapp to compress executable — docker also compresses that layer to comparable size, but compressing the executable saves about 6s of docker layer loading time. (And I also on client side used Fulcro df/load! {:parallel true} to speed up getting more info on screen.) In short, I’m super happy with that result!!! You can see how fast it loads here: https://feedly-reader-exe-rhpg5b3znq-uw.a.run.app/

😻 3
🚀 2
👍 1
Jakub Holý (HolyJak)08:12:51

Amazing! (And good you found distroless, I wanted to point it out but forgot 😅 )

borkdude08:12:18

Congrats 🎉 I bet you can trim the image size down even more. Alpine is also an option here, but this is a good start :-D

borkdude08:12:46

GraalVM also supports a dashboard mode so you can see where parts of the image size come from

Jakub Holý (HolyJak)08:12:10

I see that the base-debian11 has 20MB while static-debian11 has reportedly 2MB, if you managed to compile the graalvm binary statically so it would not need libc and openssl. Though the question is how much bigger the binary then becomes.

borkdude08:12:01

Yeah, do they also support alpine? The GraalVM image itself can also shrink considerably by being more mindful about dependencies, which means patching fulco in user space to elide the spec stuff, which I've already done to avoid the thread start in the native image

borkdude08:12:45

Native image supports threads, just not during compilation so top level core async threads had to be removed

Jakub Holý (HolyJak)08:12:05

No, distroless does not have an alpine version

borkdude09:12:15

I guess distroless comes close to the size of alpine. Then it's just the graalvm image size to optimize still

Hukka09:01:57

@U6VPZS1EK Thanks for the heads up, I probably wouldn't have noticed this otherwise (happy holidays!). 2 seconds is probably within our pain tolerance for a cold start of our API. I have to revisit this when our pilots really start, keeping a close eye on throughput vs JVM. As mentioned before, we only have CLI tools containerized and gaalified, so I haven't yet checked what is the "hello world" launch time (i.e. the baseline) for Cloud Run cold starts.

borkdude09:01:34

Usually a graalvm binary starts within a few milliseconds (10-30). Add the base cold start time of Google Cloud run: x ms. Then everything what comes on top is on you as an application developer.

genekim18:01:25

FWIW, for my non-GraalVM native image Fulcro uberjars in Google Cloud Run, I’m seeing 20-30 second cold starts. Which is “fine” for me right now, but it is something I plan on looking into exploring native images as well in the next month or two.

genekim18:01:04

(And it also occurred to me over the holidays. Because with GCR, you only pay for the time a request is being processed, sending a “keepalive” request every 5 minutes would have nearly negligible cost. Which makes a standard uberjar just fine, as it will never have a cold start…. That’s just conjecture on my part: I haven’t actually done this yet, but it would take 15m to set up something like this: • create /keepalive REST endpoint • set up Google Cloud Scheduler to hit that endpoint every 10m

Hukka18:01:28

The fake requests to keep ephemeral resources warm is certainly a proven trick. IIRC the Cloud Run instances are kept up for 15 minutes, so even fewer would work. Then again, somehow I think that if that kind of optimization is needed, there are some fundamental problems with the costs and income of your thingy 😉. It is possible to just pay for a reserved spot without playing catch with the hosting provider's sysadmins and rules lawyers.

Hukka18:01:43

The most intriguing idea I've had so far is to create a docker image that has both the graal and JVM versions within, with an additional load balancer that routes requests to the graal version at first and then moves on to the JVM when it's up. For best of both worlds. Surely the bigger size will have some impact on startup times, but I guess quite slight

❤️ 1
genekim19:01:46

Ha! Yes, this is for a personal app that only I use. In fact, due to the way the Trello API works, I’m not even sure it would work if two people were using it (rate limits would always be hit.). So, technically, it’s all cost. 🙂

borkdude19:01:52

Speaking of the cloud, I experimented with #nbb on lambda. Just a few lines of code: https://blog.michielborkent.nl/aws-lambda-nbb.html Cold start: 500ms. After that 100ms response time. This is a deployed lambda. It has a counter. I haven't measured, but after a while the counter is back to 1. https://x1gaenan7a.execute-api.eu-central-1.amazonaws.com/default/nbb_lambda_demo