Fork me on GitHub
#graalvm
<
2020-03-31
>
alekcz15:03:31

Hello. Does anyone have any tips on how to speed up my native-image build time

borkdude15:03:05

@alekcz360 Can you tell more about your current project? What kind of code / dependencies, what arguments do you pass to native-image? Etc?

alekcz15:03:24

:dependencies [ [org.clojure/clojure "1.9.0"]
                  [org.clojure/tools.cli "1.0.194"]
                  [org.martinklepsch/clj-http-lite "0.4.3"]
                  [compojure "1.6.1"]
                  [borkdude/sci "0.0.13-alpha.12"]
                  [http-kit "2.3.0"]
                  ;scgi
                  [com.climate/claypoole "1.1.4"]
                  ;includes for hosted environemnt
                  [cheshire "5.9.0"]
                  [de.ubercode.clostache/clostache "1.4.0"]
                  [hiccup "1.0.5"]
                  [seancorfield/next.jdbc "1.0.409"]
                  [org.postgresql/postgresql "42.2.11"]
                  [honeysql "0.9.10"]
                  [org.martinklepsch/clj-http-lite "0.4.3"]
                ]

:native-image {:name     "pcp"
                 :jvm-opts ["-Dclojure.compiler.direct-linking=true"]
                 :opts     ["--enable-url-protocols=http,https"
                            "--report-unsupported-elements-at-runtime"
                            "--no-fallback"
                            "--initialize-at-build-time"
                            "--allow-incomplete-classpath"
                            "--initialize-at-run-time=org.postgresql.sspi.SSPIClient"
                            "--enable-all-security-services"
                            "--no-server"
                            "-H:ReflectionConfigurationFiles=reflect-config.json"]}

alekcz15:03:04

I'm aiming to have an alternative to PHP-FPM that run sci scripts

borkdude15:03:11

Sometimes GraalVM takes a lot longer when it has just barely enough memory. Do you use an -Xmx setting?

alekcz15:03:53

I gave up on Netty and handwrote my TCP connections. Netty made the build very long.

alekcz15:03:16

Yeah locally -Xmx is acceptable. It's set to 12g 🙈

alekcz15:03:37

That doesn't work on github actions. Haha. The runners have 7GB ram

borkdude15:03:11

Yeah, I'm also hitting the limit on CircleCI a lot lately (4gb, now migrated to Github actions...)

borkdude15:03:26

Avoiding reflection and dependencies is usually also easier on GraalVM. This is a project which also uses postgres with Graal native: https://github.com/leafclick/pgmig

borkdude15:03:56

I don't have any specific recommendations beyond this. Maybe try bisecting a bit, to see what causes 📈

alekcz15:03:53

Thanks @borkdude I'll report back if I make any progress

borkdude15:03:09

Sounds like an exciting project btw. I think @jeroenvandijk also had a similar idea with sci

jeroenvandijk16:03:23

Thanks @borkdude. @alekcz360 I’ve created a poc for a php like interpreter with Sci here https://gist.github.com/jeroenvandijk/fb4cafb5022cead538cde1c7c6e39c4d Not sure if I left it in a finished state. I’m still working on this idea

👍 4
alekcz16:03:48

@borkdude @jeroenvandijk my repo is here https://github.com/alekcz/pcp. It's still a mess Seems to work well. I just need to get the pipeline working so I can test in an actual ubuntu server.

💪 4
❤️ 4
👍 4
alekcz16:03:01

@jeroenvandijk ooh that looks really cool. It goes a level deeper than mine. I'll keep an eye on that gist.

jeroenvandijk17:03:55

@alekcz360 I only wrote the php parsing, it doesn’t serve anything yet (as you can see I think). I was thinking it could make Clojure as accessible as php. Your work is already more useful as it actually serves something 🙂 What I like about php is partially it’s accessibility (just edit an html-like file), but mostly the upload and refresh feature. Sci makes this possible for Clojure. This is something i’m exploring further

borkdude17:03:32

to make it more light weight maybe you can use nginx which fires off the executable and outputs the result, so you can keep the webserver out of the executable

borkdude17:03:49

but since you're going to need a server anyway, why not use the JVM?

jeroenvandijk18:03:34

For production usage the JVM might give better performance. For local development having a server that quickly starts has some benefits in the user experience I think (especially for non-clojure devs). But then it should not depend on nginx maybe

alekcz21:03:48

@borkdude yip. The webserver is out for now. The aim is to let people use the same tool to test locally.

alekcz21:03:44

Turns out the culprit was hiccup. Once I stripped it out. It was basically christmas.

borkdude21:03:59

huh... I didn't expect hiccup to be the problem. I wonder how bootleg deals with hiccup: it's also a graalvm project: https://github.com/retrogradeorbit/bootleg

borkdude21:03:24

I never tested the hiccup library myself with graalvm, it also doesn't appear in this list: https://github.com/lread/clj-graal-docs/blob/master/doc/external-resources.md#libraries-compatible-with-graalvm

borkdude21:03:06

maybe @retrogradeorbit can chip in tomorrow (he's probably asleep now) on how he did hiccup for bootleg

borkdude21:03:19

hmm, this article does mention hiccup with graalvm: https://www.innoq.com/en/blog/native-clojure-and-graalvm/