Fork me on GitHub
#graalvm
<
2022-05-19
>
grounded_sage03:05:53

Hey everyone. I'm curious what is involved to get Datahike compiling on the main branch. Does anyone know how I would pick apart this error?

Fatal error: org.graalvm.compiler.debug.GraalError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image runtime.  To see how this object got instantiated use --trace-object-instantiation=java.lang.Thread. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
	at com.oracle.graal.pointsto.util.AnalysisFuture.setException(AnalysisFuture.java:49)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:269)
	at com.oracle.graal.pointsto.util.AnalysisFuture.ensureDone(AnalysisFuture.java:63)
	at com.oracle.graal.pointsto.heap.ImageHeapScanner.lambda$postTask$9(ImageHeapScanner.java:611)
	at com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
	at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image runtime.  To see how this object got instantiated use --trace-object-instantiation=java.lang.Thread. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
	at com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.error(DisallowedImageHeapObjectFeature.java:173)
	at com.oracle.svm.core.image.DisallowedImageHeapObjects.check(DisallowedImageHeapObjects.java:74)
	at com.oracle.svm.hosted.image.DisallowedImageHeapObjectFeature.replacer(DisallowedImageHeapObjectFeature.java:149)
	at com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:582)
	at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:257)
	at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:228)
	at com.oracle.svm.hosted.heap.SVMImageHeapScanner.transformFieldValue(SVMImageHeapScanner.java:126)
	at com.oracle.graal.pointsto.heap.ImageHeapScanner.onFieldValueReachable(ImageHeapScanner.java:331)
	at com.oracle.graal.pointsto.heap.ImageHeapScanner.lambda$createImageHeapObject$3(ImageHeapScanner.java:272)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	... 10 more

borkdude08:05:05

This means that a thread is started on the top level. E.g.:

(def x (future :hello))
triggers such an error. Have you tried following up on the suggestion in the error message?

grounded_sage08:05:30

yes I tried doing --initialize-at-run-time=... but think I am not selecting the classes properly. When I use the library https://clojars.org/com.github.clj-easy/graal-build-time I get a similar recommendation to initialize the class at runtime.

Error: Classes that should be initialized at run time got initialized during image building:
org.eclipse.jetty.util.Jetty was unintentionally initialized at build time. To see why org.eclipse.jetty.util.Jetty got initialized use --trace-class-initialization=org.eclipse.jetty.util.Jetty

borkdude08:05:34

You should add --initialize-at-build-time=org.eclipse.jetty if you are instantiating jetty on the top level

borkdude08:05:42

or push that to a delay or function

borkdude08:05:54

why do you need jetty btw?

grounded_sage08:05:26

running a server when starting the native executable

borkdude08:05:12

tell me what this has to do with datahike? try to use the least amount of dependencies possible when getting started with a native image

borkdude08:05:40

if you want a server that works out of the box with graalvm native image, try org.httpkit.server

grounded_sage08:05:05

I thought that jetty was supported with graalvm native image. https://github.com/clj-easy/graalvm-clojure/tree/master/ring-jetty

borkdude08:05:39

the example in the graal-clojure repo uses --initialize-at-build-time which is now deprecated, you need to include the full list of package prefixes

borkdude08:05:59

but you can cheat with --initialize-at-build-time=. - don't tell anyone ;)

😂 1
grounded_sage08:05:01

yea I am trying to get datahike compiling using native image the server I am also interested in / partly also using the server to easily test it.

grounded_sage08:05:19

the graal-build-time makes it default to that again right? so you would then only have to specify the runtime ones?

borkdude08:05:51

no, graal-build-time only specifies clojure packages, you still have to specify all Java packages yourself

borkdude08:05:19

in general you should try to keep the list as small as possible

grounded_sage08:05:21

oh! and jetty is not

grounded_sage08:05:16

so I am basically using Jetty because that is the recommended starter by Eric Normand and http-kit was in the give it a pass. https://ericnormand.me/mini-guide/clojure-web-servers

borkdude08:05:52

Don't just believe everything people say :)

borkdude08:05:07

It depends on your use case

borkdude08:05:36

But you can continue with jetty if you want of course, just add those arguments

grounded_sage08:05:43

haha. Yea I have had trouble finding proper comparisons of the different servers.

grounded_sage08:05:08

so this --initialize-at-build-time=org.eclipse.jetty works though I am not sure how to get this to work for superv.async

borkdude08:05:25

what is superv.async

grounded_sage08:05:15

the first one is just the org from the ring-jetty-adapter . Doing that with io.replikativ doesn't make much sense because there is also konserve etc. It says to do it with the class name. But when I look in the target directory classes I can't see how the jetty one maps to anything there.

grounded_sage08:05:54

It's a library that provides Erlang style error handling. https://github.com/replikativ/superv.async

grounded_sage08:05:21

I'm still getting that build error for some reason.

borkdude08:05:36

are you using the correct version?

borkdude08:05:54

and what error exactly?

grounded_sage08:05:19

yea I have been working up and down the stack. Even just trying to run superv.async

grounded_sage08:05:44

the one of top level of thread here. The thread error

borkdude08:05:10

That indicates that a thread is being started from the top level. Maybe the native image check isn't working anymore

grounded_sage08:05:54

I'm not using it actually.

grounded_sage08:05:06

;; a simple global instance, will probably be removed
(def S
  (try
    ;; We cannot run the simple-supervisor thread in a static context inside
    ;; native image.
    (dummy-supervisor)
    #_#?(:clj (if (native-image-build?)
                (dummy-supervisor)
                (simple-supervisor))
         :cljs (simple-supervisor))
    (catch #?(:clj Exception :cljs js/Error) _
      (dummy-supervisor) #_(simple-supervisor))))

borkdude08:05:23

yes, but it will still be used simply because it's a top level thing

grounded_sage08:05:29

I commented it out to just use the dummy-supervisor which was added for the native image support

borkdude08:05:15

maybe you can comment out more until you don't get the error and then detect where the thread is being started

👍 1
grounded_sage08:05:01

Yea I was just trying to make it work quickly because working on getting it right isn't a major priority atm. So was hoping I could just initialise the class at runtime.

borkdude08:05:39

anything used from clojure cannot be initialized at runtime, unless you push it to a function (or delay which is essentially a function)

grounded_sage09:05:42

okay thats a valuable insight. Thanks. Perhaps I will try and pick apart superv.async a bit more and find where this issue is.

simongray08:05:38

Does anyone here know why VisualVM also exists bundled with GraalVM? The website doesn’t really explain the split. https://visualvm.github.io/

borkdude08:05:34

Perhaps just for convenience?

borkdude08:05:57

Or perhaps their architecture requires specific changes to it? Don't know. You could ask on their dedicated slack

simongray08:05:33

yeah, maybe I will do that.

littleli15:05:57

My suggestion is that split is required because of different JIT engines in charge between openjdk and graalvm which may require additional treatment from the observability point of view.

littleli15:05:46

C1, C2 vs Graal (?)