Fork me on GitHub
#graalvm
<
2020-08-04
>
sergey16:08:35

has anyone here gotten the cognitect aws library to run on a native-image build? (specifically the s3 client)

borkdude20:08:04

That must be the most requested native-image library right now

✔️ 3
sergey00:08:28

ooh there's some interesting workarounds in that thread; thanks for the share!

sergey00:08:40

I've been playing around with adding the cognitect s3 client to to a native-image cli app and ended up getting blocked w/ a runtime Unsupported method java.lang.ClassLoader.defineClass1. (this was a bit before I saw the graalvm issue on the cognitect repo)

jeroenvandijk06:08:58

@U0180E1CB62 The defineClass1 error is very generic and it doesn’t point to one cause in my experience (e.g. it could be eval, but also something less dynamic). If you want to check where it breaks, probably good to start bottom up and keep adding stuff until you find the issue. Here is a start https://github.com/AdGoji/aws-api/tree/master/src/adgoji/cognitect/aws (forked some time ago though)

jeroenvandijk07:08:27

@U04V15CAJ I think it’s popular because of the possibility to have one interface to aws. Although the python cli tool is pretty user friendly, the installation is a bit fragile as it’s depending on specific python versions (or libs). I have had to reinstall the aws cli tool several times due to a change in underlying system (not the aws api). For me this causes unnecessary work and anxiety about my setup. So I would definitely welcome additional help here :)

sergey18:08:08

gotcha, yea the full stack trace was:

Exception in thread "main" Syntax error compiling fn* at (cognitect/aws/http/cognitect.clj:1:1).
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:7115)
	at clojure.lang.Compiler.analyze(Compiler.java:6789)
	at clojure.lang.Compiler.eval(Compiler.java:7174)
	at clojure.lang.Compiler.eval(Compiler.java:7166)
	at clojure.lang.Compiler.load(Compiler.java:7636)
	at clojure.lang.RT.loadResourceScript(RT.java:381)
	at clojure.lang.RT.loadResourceScript(RT.java:372)
	at clojure.lang.RT.load(RT.java:459)
	at clojure.lang.RT.load(RT.java:424)
	at clojure.core$load$fn__6844.invoke(core.clj:6128)
	at clojure.core$load.invokeStatic(core.clj:6127)
	at clojure.core$load.doInvoke(core.clj:6111)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5910)
	at clojure.core$load_one.invoke(core.clj:5905)
	at clojure.core$load_lib$fn__6784.invoke(core.clj:5950)
	at clojure.core$load_lib.invokeStatic(core.clj:5949)
	at clojure.core$load_lib.doInvoke(core.clj:5930)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$load_libs.invokeStatic(core.clj:5987)
	at clojure.core$load_libs.doInvoke(core.clj:5971)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$require.invokeStatic(core.clj:6009)
	at cognitect.aws.dynaload$load_ns.invokeStatic(dynaload.clj:5)
	at cognitect.aws.dynaload$load_var.invokeStatic(dynaload.clj:12)
	at cognitect.aws.http$resolve_http_client.invokeStatic(http.clj:78)
	at cognitect.aws.client.shared$fn__11224.invokeStatic(shared.clj:9)
	at cognitect.aws.client.shared$fn__11224.invoke(shared.clj:9)
	at clojure.lang.Delay.deref(Delay.java:42)
	at clojure.core$deref.invokeStatic(core.clj:2322)
	at cognitect.aws.client.shared$http_client.invokeStatic(shared.clj:17)
	at cognitect.aws.client.api$client.invokeStatic(api.clj:76)
	at shatest.local_sha$list_buckets.invokeStatic(local_sha.clj:8)
	at shatest.local_sha$_main.invokeStatic(local_sha.clj:13)
	at shatest.local_sha$_main.doInvoke(local_sha.clj:13)
	at clojure.lang.RestFn.invoke(RestFn.java:397)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at shatest.local_sha.main(Unknown Source)
Caused by: com.oracle.svm.core.jdk.UnsupportedFeatureError: Unsupported method java.lang.ClassLoader.defineClass1(String, byte[], int, int, ProtectionDomain, String) is reachable
	at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:86)
	at java.lang.ClassLoader.defineClass1(ClassLoader.java)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:635)
	at clojure.lang.DynamicClassLoader.defineClass(DynamicClassLoader.java:46)
	at clojure.lang.Compiler$ObjExpr.getCompiledClass(Compiler.java:4986)
	at clojure.lang.Compiler$FnExpr.parse(Compiler.java:4116)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:7105)
	... 40 more
all that I'm doing is calling :ListBuckets from the cognitect s3 library. I compiled the native-image exe after generating assisted config files for an uberjar w/ the same code (described https://www.graalvm.org/docs/Native-Image/user/CONFIGURE)

borkdude19:08:06

Yeah, it's trying to load a namespace dynamically, that won't work

borkdude19:08:35

Maybe it helps when you require that namespace as part of your program so dynaload does not load it

borkdude19:08:47

Also use direct linking if you're not doing that already

sergey21:08:15

hmm how do I require the namespace so that it doesn't get loaded via dynaload?

sergey21:08:15

(I'm already compiling with -Dclojure.compiler.direct-linking=true )

borkdude21:08:00

just require it by hand somewhere top-level

sergey21:08:12

gotcha, will give it a shot and report back