Fork me on GitHub
#graalvm
<
2021-05-04
>
Karol Wójcik15:05:02

Java 17 is dropping support for AOT 😮 How we will compile Clojure programs with GraalVM?

borkdude15:05:18

This doesn't have to do anything with Clojure AOT. Clojure AOT is just regular byte code generation, like javac does with .java programs.

borkdude15:05:40

It seems they are removing some experimental tools like jaotc which isn't used by anyone in production.

👍 3
Karol Wójcik15:05:00

Good to know 🙂 Thank you

Karol Wójcik15:05:06

What may cause this:

Fatal <error:com.oracle.svm.core.util.VMError>$HostedError: com.oracle.svm.core.util.VMError$HostedError: fierycod.holy_lambda.util$call.invokeStatic(Object, Object): has no code address offset set.
It's the first time I'm seeing it

borkdude15:05:36

I have seen this a couple of times. Are you perhaps compiling with leiningen?

Karol Wójcik15:05:59

Nope. With depstar. On leiningen I have never seen it. Actualy I've switched from leiningen to depstar and then bizzare things with AOT started to happen.

borkdude15:05:44

@ericdallo had it too with clojure-lsp. After switching from lein to regular (compile ') it went away

borkdude15:05:28

you can post your uberjar and make an issue at oracle/graal about it

ericdallo15:05:29

Yeah, I had the same and I'm not remembering how we fixed it 😅, but I think it was indeed related with lein

borkdude15:05:51

it was "fixed" by changing from lein aot to regular (compile ')

👍 3
ericdallo15:05:59

Oh, yeah, I needed to migrate to depstar to fix that

Karol Wójcik15:05:49

It's some random bug 😄 I'm using depstar and I've just purged all the classes, recompiled three times and on third Graalvm compilation was successful. Btw are you using native configuration to produce executable?

Tomas Brejla16:05:56

Hello. I believe I've heard in defn podcast #50 with borkdude that there's some flag in graalvm's native-image that "forgives" potential usage of reflection. You might want to try using that flag if there's a chance that such code doesn't actually get executed in your specific runtime path. Does such flag rings a bell? Which one was that? (it's a pitty that it's not easy to "grep" a podcast episode 🙂)

Tomas Brejla16:05:57

Thanks @borkdude. I tried using reflection.json that gets created in babashka native image compilation as an example and stripped it down to just java.net.URLEncoder and java.net.URLDecoder. I've chosen these 2 because lein uberjar gives me following reflection warnings:

Reflection warning, clojure/data/xml/jvm/name.clj:35:1 - call to static method decode on java.net.URLDecoder can't be resolved (argument types: unknown, java.lang.String).
Reflection warning, clojure/data/xml/jvm/name.clj:38:1 - call to static method encode on java.net.URLEncoder can't be resolved (argument types: unknown, java.lang.String).
But even with following reflection.json
[{
        "name": "java.net.URLEncoder",
        "allPublicMethods": true,
        "allPublicFields": true,
        "allPublicConstructors": true
    },
    {
        "name": "java.net.URLDecoder",
        "allPublicMethods": true,
        "allPublicFields": true,
        "allPublicConstructors": true
    }
]
.. I'm still not able to create a native-only no-fallback image:
native-image -H:ReflectionConfigurationFiles=reflection.json --no-fallback -jar target/uberjar/myapp-0.1.0-SNAPSHOT-standalone.jar

Error: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method java.lang.ClassLoader.defineClass(String, byte[], int, int) is reachable

borkdude16:05:37

@brdloush Take a look at the other graalvm arguments in babashka. You need a few others.

borkdude16:05:12

You need at least:

--initialize-at-build-time

borkdude16:05:43

And sometimes --report-unsupported-elements-at-runtime

borkdude16:05:07

It depends on what you are doing. If you want to rely on eval then that's not going to work

Tomas Brejla16:05:17

Well it's not my code which is causing the trouble I guess. I believe those clojure/data/xml/jvm/name.clj:35:1 warnings ^^^ comes from the fact that I'm using one of these libraries:

[com.cognitect.aws/api "0.8.505"]
[com.cognitect.aws/endpoints "1.1.11.1001"]
[com.cognitect.aws/s3 "811.2.889.0"]
[com.cognitect.aws/iam "811.2.889.0"]
[com.cognitect.aws/sts "811.2.889.0"]                 
[throttler "1.0.0"]
[org.clojure/tools.cli "1.0.206"] 
I think it's coming from those cognitext.aws which internally use data.xml, which might be causing the native compilation issue.

borkdude16:05:51

clojure.data.xml works quite well with graalvm, I'm also using it in babashka

👍 3
borkdude16:05:12

it's actually the aws library itself which is problematic since it uses quite a lot of dynamics

borkdude16:05:42

but we (me and some others) have managed to "natify" that as well: https://github.com/babashka/pod-babashka-aws

❤️ 3
Tomas Brejla16:05:46

Sounds like I might even switch to babashka for this utility I'm building then 😄

borkdude16:05:09

Here are some hints to what is important to get it working with graalvm: https://github.com/BrunoBonacci/graalvm-clojure/tree/master/aws-api-s3

Tomas Brejla16:05:20

oooh,.... reading the README.md : • Credentials: custom flows are supported, but not by extending CredentialsProvider interface. See https://github.com/babashka/pod-babashka-aws#credentials for options. This might be the issue, I'm actually using (reify credentials/CredentialsProvider that's very likely the problem.

borkdude16:05:58

That note only applies to the babashka aws pod, not to using aws-api as a lib in graalvm native

👏 3
Karol Wójcik16:05:57

@brdloush You can automate generation of native configuration. For instance holy-lambda has a in-agent macro which executes some code only in agent context. https://www.graalvm.org/reference-manual/native-image/BuildConfiguration/#assisted-configuration-of-native-image-builds https://github.com/FieryCod/holy-lambda/blob/master/src/fierycod/holy_lambda/agent.clj#L21 All you need is: 1. Add some in-context calls which utilizes aws-api 2. Uberjar application 3. Generate all configs which should be passed to native-image

java -agentlib:native-image-agent=config-output-dir=resources/native-configuration \
			           -Dexecutor=native-agent \
			           -jar target/output.jar

Tomas Brejla16:05:57

Thanks a lot to both of you guys. Right now I'll continue adding features to the uberjar-only version that needs the jvm, but later I'll definitely give this a try. I was just curious whether I'll be able to easily native-compile that app, but it seems that it needs some non-trivial effort at the moment.