holy-lambda 2022-05-09 | Slack Archive

steveb8n07:05:36

Q: I just updated to the latest HL. I migrated all the :envs out and was happy to see much less mem required for native executable. but now my native lambdas fail with this error: Caused by: java.io.FileNotFoundException: Could not locate clojure/core/server__init.class, clojure/core/server.clj or clojure/core/server.cljc on classpath Has anyone seen this before?

steveb8n07:05:19

I’ve tried a fresh HL project. it creates a native executable ok but the sam local invoke fails. not sure why. that means I can’t compare with a working sample to find the issue

steveb8n07:05:32

I have no clj code that requires or invokes “server” so I’m a bit confused

steveb8n08:05:37

ah, I didn’t have the clj-easy dep included. working my way through that now

Karol Wójcik08:05:14

Karol Wójcik08:05:35

Yeah. Exactly! clj-easy is what you need ;>

steveb8n08:05:08

thanks. just learning how to do the init at runtime thing now with this new setup

❤️ 1

Karol Wójcik08:05:34

Cool. I hope you’re already using ARM ;)

steveb8n08:05:12

no, still on intel. I don’t need the extra speed

steveb8n08:05:28

might switch when I get more load in future

steveb8n09:05:19

Q: I have some AWS api v2 classes that need to be runtime init. I have add the graal flag but do I also need to move them out of the namespace import form and import them at runtime when they are used in fns now? @UJ1339K2B

steveb8n09:05:41

at the moment the ns import is causing them to init at build time

steveb8n09:05:15

I’m not sure how to do a runtime import in clj

steveb8n09:05:38

I suspect I need to use hl:native:conf for this, like in v0.5.0

Karol Wójcik09:05:49

You can set this class as init-at-build-time.

steveb8n09:05:21

ok. isn’t that done automatically by clj-easy?

Karol Wójcik09:05:32

Nope.

Karol Wójcik09:05:58

clj-easy is for adding Clojure namespaces to build-time.

steveb8n09:05:19

ok. I’ll try that. so 0.6.0 should not require the use of hl:native:conf at all now?

steveb8n09:05:41

the docs seem to imply that it’s still useful so I’m not sure

Karol Wójcik09:05:42

Why you thing configuration is not necessary?

Karol Wójcik09:05:57

It's still necessary to run this step.

steveb8n09:05:06

ok good to know

steveb8n09:05:38

the reason I though it was not required was because of the sample project. in there, conf is not invoked

steveb8n09:05:11

but this is good. I am familiar will conf so I’ll start using it tomorrow. stopping for the night now

steveb8n09:05:15

thanks for the pointers

Karol Wójcik09:05:26

This is because HL now ships with the default configuration for it's internal namespaces.

Karol Wójcik09:05:03

But other namespaces/java classes may still require generating configuration.

steveb8n03:05:22

Hi @UJ1339K2B I’m still having trouble making this work in CI. I can make it work on M1 and Intel locally but not on CI where I need it most. I suspect that it’s a versioning problem. The local good builds use jdk11 for conf and executable but, in CI, it uses jdk8 for the executable step

steveb8n03:05:24

+ bb hl:native:executable [holy-lambda] Executable /root/.graalvm/bin/java does not exists! Using fallback: java instead! Did you set up GRAALVM_HOME? [holy-lambda] Executable /root/.graalvm/bin/native-image does not exists! Using fallback: native-image instead! Did you set up GRAALVM_HOME? [holy-lambda] Command <hl:native:executable> Executing [ /opt/graalvm-ce-java8-21.2.0/jre/bin/java \ -XX:+UseParallelGC \ -XX:+UnlockExperimentalVMOptions \ -XX:+EnableJVMCI \

steveb8n03:05:23

this is my best guess at why it’s failing there. my CI build uses fierycod/graalvm-native-image:ce as its base image. could this be the reason?

steveb8n03:05:35

ok. just as I typed this, I got it to compile. not sure how I found the magic combination. was not very scientific

steveb8n03:05:54

any suggestions about that native:executable version?

steveb8n03:05:21

also: since I’m using an HL docker image, could/should I use HL_NO_DOCKER?

Karol Wójcik04:05:43

To a last question: YES. Regarding image. The CE is deprecated builder. You can find most recent builders images here https://github.com/FieryCod/holy-lambda/pkgs/container/holy-lambda-builder

steveb8n04:05:55

thanks. I tried one of those images but it didn’t have the “clojure” cli in it. I guess I need to look closer

steveb8n04:05:04

and I’ll try no docker as well

steveb8n04:05:18

would you consider adding the “clojure” cli to those images so I can run tests using the same image? I can create my own image based on those but it’s simpler if I just have 1

steveb8n05:05:58

maybe that’s not possible due to EE license constraints?

steveb8n05:05:12

I’m creating my own image from yours

Karol Wójcik05:05:56

Nah. I dont want to add Clojure to an image. In future I will probably add two variant of images. One extra slim based on alpine and without polyglot capabilities, and one with. For CI you should treat the builder images as a base.

steveb8n06:05:29

fair enough. I can confirm that with a CI docker image derived from your stable release, the versions now match for “conf” and “executable”. one less variable to worry about

steveb8n06:05:27

and a lot less memory consumed. so much better. thanks

Karol Wójcik06:05:46

Karol Wójcik06:05:55

Very happy to hear it!

Volodymyr Vizovskyy17:05:01

Hi! I just followed https://github.com/FieryCod/holy-lambda/tree/master/examples/bb/native/cognitect.aws.api guide to set up an AWS client in my lambda. However, when I try to bb hl:native:executablethis is what I see:

Error: Unsupported features in 4 methods
Detailed message:
Error: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image runtime.  To see how this object got instantiated use --trace-object-instantiation=java.lang.Thread. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Trace: Object was reached by
	reading field clojure.lang.Delay.val of
		constant [email protected] reached by
	reading field clojure.lang.Var.root of
		constant [email protected] reached by
	scanning method clojure.core.async.impl.timers$timeout.invokeStatic(timers.clj:58)
Call path from entry point to clojure.core.async.impl.timers$timeout.invokeStatic(long):
	at clojure.core.async.impl.timers$timeout.invokeStatic(timers.clj:58)
	at clojure.core.async$timeout.invokeStatic(async.clj:109)
	at clojure.core.proxy$clojure.lang.APersistentMap$ff19274a.iterator(Unknown Source)
	at clojure.lang.RT.seqFrom(RT.java:547)
	at clojure.lang.RT.seq(RT.java:537)
	at com.oracle.svm.jni.JNIJavaCallWrappers.jniInvoke_VARARGS_core_main_ac790e17c7c5fc73439035595d460119bb4bd9a0(generated:0)

Has anybody ever had anything similar?

Karol Wójcik08:05:04

I will check this.

Karol Wójcik05:06:27

@U03CM8SENKE was you able to get rid of the issue?

Volodymyr Vizovskyy21:06:25

Yes, by rewriting almost everything from the first principles 😅 it turned out that what slipped my eyes was this bad boy: https://github.com/brunoV/throttler which is, according to its own README, Based on core.async - not a known friend of GraalVM.

Karol Wójcik06:06:28

Oh cool! Yes. Unfortunately the core.async is not an easy beast to integrate with GraalVM.

Karol Wójcik06:06:37

Cool you have figured it out!

🙌 1

Volodymyr Vizovskyy17:05:30

full trace here btw. It's HUGE

2022-05-09

Channels