Clojurians Log v2
holy-lambda
Q: I just updated to the latest HL. I migrated all the :envs out and was happy to see much less mem required for native executable. but now my native lambdas fail with this error: Caused by: java.io.FileNotFoundException: Could not locate clojure/core/server__init.class, clojure/core/server.clj or clojure/core/server.cljc on classpath Has anyone seen this before?
I’ve tried a fresh HL project. it creates a native executable ok but the sam local invoke fails. not sure why. that means I can’t compare with a working sample to find the issue
I have no clj code that requires or invokes “server” so I’m a bit confused
ah, I didn’t have the clj-easy dep included. working my way through that now
:D
Yeah. Exactly! clj-easy is what you need ;>
thanks. just learning how to do the init at runtime thing now with this new setup
Cool. I hope you’re already using ARM ;)
no, still on intel. I don’t need the extra speed
might switch when I get more load in future
Q: I have some AWS api v2 classes that need to be runtime init. I have add the graal flag but do I also need to move them out of the namespace import form and import them at runtime when they are used in fns now? @karol.wojcik
at the moment the ns import is causing them to init at build time
I’m not sure how to do a runtime import in clj
I suspect I need to use hl:native:conf for this, like in v0.5.0
You can set this class as init-at-build-time.
ok. isn’t that done automatically by clj-easy?
Nope.
clj-easy is for adding Clojure namespaces to build-time.
ok. I’ll try that. so 0.6.0 should not require the use of hl:native:conf at all now?
the docs seem to imply that it’s still useful so I’m not sure
Why you thing configuration is not necessary?
It's still necessary to run this step.
ok good to know
the reason I though it was not required was because of the sample project. in there, conf is not invoked
but this is good. I am familiar will conf so I’ll start using it tomorrow. stopping for the night now
thanks for the pointers
This is because HL now ships with the default configuration for it's internal namespaces.
But other namespaces/java classes may still require generating configuration.
Hi @karol.wojcik I’m still having trouble making this work in CI. I can make it work on M1 and Intel locally but not on CI where I need it most. I suspect that it’s a versioning problem. The local good builds use jdk11 for conf and executable but, in CI, it uses jdk8 for the executable step
+ bb hl:native:executable
[holy-lambda] Executable /root/.graalvm/bin/java does not exists! Using fallback: java instead! Did you set up GRAALVM_HOME?
[holy-lambda] Executable /root/.graalvm/bin/native-image does not exists! Using fallback: native-image instead! Did you set up GRAALVM_HOME?
[holy-lambda] Command <hl:native:executable>
Executing [
/opt/graalvm-ce-java8-21.2.0/jre/bin/java \
-XX:+UseParallelGC \
-XX:+UnlockExperimentalVMOptions \
-XX:+EnableJVMCI \
this is my best guess at why it’s failing there. my CI build uses fierycod/graalvm-native-image:ce as its base image. could this be the reason?
ok. just as I typed this, I got it to compile. not sure how I found the magic combination. was not very scientific
any suggestions about that native:executable version?
also: since I’m using an HL docker image, could/should I use HL_NO_DOCKER?
To a last question: YES. Regarding image. The CE is deprecated builder. You can find most recent builders images here https://github.com/FieryCod/holy-lambda/pkgs/container/holy-lambda-builder
thanks. I tried one of those images but it didn’t have the “clojure” cli in it. I guess I need to look closer
and I’ll try no docker as well
would you consider adding the “clojure” cli to those images so I can run tests using the same image? I can create my own image based on those but it’s simpler if I just have 1
maybe that’s not possible due to EE license constraints?
I’m creating my own image from yours
Nah. I dont want to add Clojure to an image. In future I will probably add two variant of images. One extra slim based on alpine and without polyglot capabilities, and one with. For CI you should treat the builder images as a base.
fair enough. I can confirm that with a CI docker image derived from your stable release, the versions now match for “conf” and “executable”. one less variable to worry about
and a lot less memory consumed. so much better. thanks
:>
Very happy to hear it!
Hi! I just followed https://github.com/FieryCod/holy-lambda/tree/master/examples/bb/native/cognitect.aws.api guide to set up an AWS client in my lambda. However, when I try to bb hl:native:executablethis is what I see:
Error: Unsupported features in 4 methods
Detailed message:
Error: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image runtime. To see how this object got instantiated use --trace-object-instantiation=java.lang.Thread. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Trace: Object was reached by
reading field clojure.lang.Delay.val of
constant clojure.lang.Delay@a7c489a reached by
reading field clojure.lang.Var.root of
constant clojure.lang.Var@296a969a reached by
scanning method clojure.core.async.impl.timers$timeout.invokeStatic(timers.clj:58)
Call path from entry point to clojure.core.async.impl.timers$timeout.invokeStatic(long):
at clojure.core.async.impl.timers$timeout.invokeStatic(timers.clj:58)
at clojure.core.async$timeout.invokeStatic(async.clj:109)
at clojure.core.proxy$clojure.lang.APersistentMap$ff19274a.iterator(Unknown Source)
at clojure.lang.RT.seqFrom(RT.java:547)
at clojure.lang.RT.seq(RT.java:537)
at com.oracle.svm.jni.JNIJavaCallWrappers.jniInvoke_VARARGS_core_main_ac790e17c7c5fc73439035595d460119bb4bd9a0(generated:0)I will check this.
Yes, by rewriting almost everything from the first principles 😅 it turned out that what slipped my eyes was this bad boy: https://github.com/brunoV/throttler which is, according to its own README, Based on core.async - not a known friend of GraalVM.
Oh cool! Yes. Unfortunately the core.async is not an easy beast to integrate with GraalVM.
Cool you have figured it out!
@volodymyr085 was you able to get rid of the issue?
full trace here btw. It's HUGE