Clojurians Log v2

It's auto generated via hl:native:conf. See https://fierycod.github.io/holy-lambda/#/native-backend Configuration is auto-added 😉 Regarding the --initialize-at-build-time. The commit with the configuration on http-kit side has not been yet released.

Try adding:

--initialize-at-build-time=org.httpkit.client.ClientSslEngineFactory\$SSLHolder

--initialize-at-build-time=org.httpkit.client.ClientSslEngineFactory$SSLHolder

to bb.edn native-image-args 🙂

Reflection != init flags

tried with the initialize at build flag. It throws this:

Error: Detected an instance of Random/SplittableRandom class in the image heap. Instances created during image generation have cached seed values and don't behave as expected.  To see how this object got instantiated use --trace-object-instantiation=java.security.SecureRandom. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Detailed message:
Trace: Object was reached by 
	reading field sun.security.ssl.SSLContextImpl.secureRandom of
		constant sun.security.ssl.SSLContextImpl$TLSContext@7f03e44d reached by 
	reading field javax.net.ssl.SSLContext.contextSpi of
		constant javax.net.ssl.SSLContext@49504750 reached by 
	scanning method org.httpkit.client.ClientSslEngineFactory.getContextInstance(ClientSslEngineFactory.java:29)
Call path from entry point to org.httpkit.client.ClientSslEngineFactory.getContextInstance(): 
	at org.httpkit.client.ClientSslEngineFactory.getContextInstance(ClientSslEngineFactory.java:29)
	at org.httpkit.client.ClientSslEngineFactory.trustAnybody(ClientSslEngineFactory.java:33)
	at org.httpkit.client$coerce_req.invokeStatic(client.clj:68)
	at org.httpkit.client$request.invokeStatic(client.clj:242)
	at org.httpkit.client$request.doInvoke(client.clj:180)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at org.singularity_group.bot_announce.core.main(Unknown Source)
	at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:146)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:182)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)

Add tracing for Secure Random and show the trace

What GraalVM version do you use?

graalvm-ce-java8-21.2.0

ah maybe it's old

No

Trace: Object was reached by 
	reading field sun.security.ssl.SSLContextImpl.secureRandom of
		constant sun.security.ssl.SSLContextImpl$TLSContext@1beff060 reached by 
	reading field javax.net.ssl.SSLContext.contextSpi of
		constant javax.net.ssl.SSLContext@1d491df0 reached by 
	scanning method org.httpkit.client.ClientSslEngineFactory.getContextInstance(ClientSslEngineFactory.java:29)
Call path from entry point to org.httpkit.client.ClientSslEngineFactory.getContextInstance(): 
	at org.httpkit.client.ClientSslEngineFactory.getContextInstance(ClientSslEngineFactory.java:29)
	at org.httpkit.client.ClientSslEngineFactory.trustAnybody(ClientSslEngineFactory.java:33)
	at org.httpkit.client$coerce_req.invokeStatic(client.clj:68)
	at org.httpkit.client$request.invokeStatic(client.clj:242)
	at org.httpkit.client$request.doInvoke(client.clj:180)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at org.singularity_group.bot_announce.core.main(Unknown Source)
	at com.oracle.svm.jni.JNIJavaCallWrappers.jniInvoke_VA_LIST:Lorg_singularity_1group_bot_1announce_core_2_0002emain_00028_3Ljava_lang_String_2_00029V(generated:0)

It's not about graalvm. It's about http-kit that stopped supporting GraalVM cc @borkdude. Here the regression was introduced: https://github.com/http-kit/http-kit/commit/1caddcf162de9ec92d320875dcb6a1a9d188f29f#diff-6b66607e20f6598724bd4dd94521a74eecafaec6c995331b3164e0ba89f1f70d

Which version of http-kit do you use? @benjamin.schwerdtner

I see. So I could try downgrading for now

Right.

http-kit/http-kit {:mvn/version "2.5.3"}

No, this is wrong. http-kit does support graalvm and you should in fact be using the newest alpha which has built-in native image config

however you should probably not be creating http clients on the top level which will initialize this class at build time, which is never supported

and that might be the case in your code.

But still if @borkdude says you should upgrade, you should upgrade 😄

to 2.6.0-alpha1 ? Toplevel means in def right but now inside functions @borkdude

Or use a delay

https://github.com/http-kit/http-kit/blob/master/resources/META-INF/native-image/http-kit/http-kit/native-image.properties

correct

huehue worked now 🙂

When run on local machine it uses just over 6Gb. The CI system claims to provide 8Gb but I suppose there is some overhead

probably because the Docker daemon needs the other 2Gb

Try setting maximum heap size up to 5-6gb.

Sorry. It should be -J-Xmx5G set in native-image-args

ok. will do. looks like I can reduce the docker daemon mem also

will try both

Don’t do it by limiting memory size of your daemon. Native image does not utilize full all the available memory. Better to set memory limit to native-image and check if the argument is there by using —verbose

0.5.0 fixed it. used about 1Gb less memory

happy days

New version will be compatible with 0.5.0, but also slightly faster and more memory efficient. ;)

great news

hmm, I need a little help with this config. I think you mean that I should use the props from the native-image.properties file in that repo. but where do I add these props? somewhere in bb.edn?

You have to add the dependency. Readme.md of the project should help you understand what you should do.

Alternatively you can pass the params from native-image.properties to bb.edn native-image-args

Do you know how to do this? @steveb8n

I’ve added the dep. and the lambdas are working and logging so it’s not a big deal

I didn’t realise the readme at the root of the project was where the instructions exist

I was looking in the example project of the link you added above

is it a straight copy from this file https://github.com/clj-easy/graal-config/blob/master/config/org.slf4j/slf4j-simple/resources/META-INF/native-image/org.slf4j/sl4fj-simple/native-image.properties into bb.edn?

Nope. native-image automatically gathers all native-image.properties and merges them together. This is not a feature of HL :D

Basically you have to add a dependency to a classpath and everything is done automatically

Anyway. Everything works fine? @steveb8n

yeah. it’s all working great

I’ve added the simple dep but still get the warning. however logging works properly so I don’t really care about the warning

I’ll leave it alone for now

What is the warning?

It's probably easy to fix with one line of configuration

hmm. I can’t find it now. maybe it’s already fixed

let’s forget it in that case. false alarm 🙂

Ok 🙂

ah found it….

+ bb native:executable --runtime native
Could not find /project/.holy-lambda/clojure-tools-1.10.3.967.jar
Attempting download from 
Checking out:  at 5eaccc18337b6d875cc81d5908432d4df57899bc
Downloading: clojure-term-colors/clojure-term-colors/0.1.0/clojure-term-colors-0.1.0.pom from clojars
Downloading: clojure-term-colors/clojure-term-colors/0.1.0/clojure-term-colors-0.1.0.jar from clojars
[holy-lambda] Executable /root/.graalvm/bin/java does not exists! Using fallback: java instead! Did you set up GRAALVM_HOME?
[holy-lambda] Executable /root/.graalvm/bin/native-image does not exists! Using fallback: native-image instead! Did you set up GRAALVM_HOME?
[holy-lambda] Command <native:executable>
Warning: Ignoring server-mode native-image argument --no-server.
[output:164]    classlist:   6,913.00 ms,  2.23 GB
--initialize-at-build-time without arguments has been deprecated and will be removed in GraalVM 22.0.
[output:164]        (cap):     651.52 ms,  2.23 GB
WARNING: Could not resolve fierycod.holy_lambda.core$entrypoint$fn__6578 for reflection configuration. Reason: java.lang.ClassNotFoundException: fierycod.holy_lambda.core$entrypoint$fn__6578.
[output:164]        setup:   2,515.74 ms,  2.23 GB
Warning: class initialization of class org.apache.commons.logging.impl.Log4JLogger failed with exception java.lang.NoClassDefFoundError: org/apache/log4j/Priority. This class will be initialized at run time because option --allow-incomplete-classpath is used for image building. Use the option --initialize-at-run-time=org.apache.commons.logging.impl.Log4JLogger to explicitly request delayed initialization of this class.
WARNING: Could not register reflection metadata for org.apache.commons.logging.impl.Log4JLogger. Reason: java.lang.NoClassDefFoundError: org/apache/log4j/Priority.
[output:164]     (clinit):   2,187.24 ms,  5.27 GB
[output:164]   (typeflow):  57,038.49 ms,  5.27 GB
[output:164]    (objects):  69,540.11 ms,  5.27 GB

strange warning about HL entrypoint as well

Ah. Right. The first one regarding entrypoint will be fixed in the next release. Ignore it for now.

Could you remove this: org.apache.commons.logging.impl.Log4JLogger and fierycod.holy_lambda.core$entrypoint$fn__6578 from reflection configurations?

Also add this option --initialize-at-run-time=org.apache.commons.logging.impl.Log4JLogger to bb.edn 😄

can add the option easily. removing those entries means extra CI complexity because I run native:conf in CI as well

I’ll meet you halfway 🙂

This one should work just fine: --initialize-at-run-time=org.apache.commons.logging.impl.Log4JLogger

yep. running that one through CI now

WARNING: Could not register reflection metadata for org.apache.commons.logging.impl.Log4JLogger. Reason: java.lang.NoClassDefFoundError: org/apache/log4j/Priority.

same for that class?

This warning you get, because :conf generates an entry for Log4JLogger, but the class is not in the classpath. You should remove the entry from reflect-configurations 😄

Roger that. Thanks

SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”. SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. WARNING: Could not register reflection metadata for org.apache.commons.logging.impl.Log4JLogger. Reason: java.lang.NoClassDefFoundError: org/apache/log4j/Priority. Warning: class initialization of class org.apache.commons.logging.impl.Log4JLogger failed with exception java.lang.NoClassDefFoundError: org/apache/log4j/Priority. This class will be initialized at run time because option --allow-incomplete-classpath is used for image building. Use the option --initialize-at-run-time=org.apache.commons.logging.impl.Log4JLogger to explicitly request delayed initialization of this class. [

Add slf4j-simple and the following configuration https://github.com/clj-easy/graal-config/tree/master/config/org.slf4j/slf4j-simple

Btw could you remove Log4JLogger from the reflection-config.json?

I’ll look into that once mem is fixed