Fork me on GitHub
#graalvm
<
2022-02-10
>
wombawomba13:02:15

I'm trying to build a project using native-image, and I'm running into a few "Detected a started Thread in the image heap." errors that seem to originate in clojure.lang.Agent.soloExecutor (which I assume is due to top-level agent calls somewhere). My project doesn't make use of agent, so these errors have to be caused by some dependency. How can I figure out which dependency is causing this, and what can I do about it?

borkdude13:02:21

@wombawomba you can find out by excluding certain dependencies from your own program and then bisect and then check the dependencies' source

borkdude13:02:41

what you can do about it: fork and/or ask the maintainer to wrap the top level thread in a delay

wombawomba13:02:18

hmm... it's a pretty complex project so just excluding random dependencies would be a lot of work

wombawomba13:02:42

surely there must be some way of figuring it out automatically?

borkdude13:02:54

another way could be to alter-var-root agent before you load anything and throw an exception in the agent call and then inspect the stacktrace :)

wombawomba13:02:09

ah, now we're talking 🙂 I'll try that

borkdude13:02:35

I hope it won't interfere with anything else while loading, but worth a try

borkdude13:02:09

another way could be to analyze all your deps with clj-kondo and then filter out the agent calls

wombawomba13:02:38

I can't imagine I'm the only person who's tried to use native-image with a project that uses clojure.tools.logging — could there be some sort of workaround here that doesn't involve patching that library?

borkdude13:02:05

nope, babashka also has tools.logging

borkdude13:02:29

let me check if I patched anything for that

wombawomba13:02:50

perhaps I'm mistaken about what's causing the error?

wombawomba13:02:27

the full error is:

Error: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image runtime.  To see how this object got instantiated use --trace-object-instantiation=java.lang.Thread. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Trace: Object was reached by
	reading field java.util.concurrent.ThreadPoolExecutor$Worker.thread of
		constant java.util.concurrent.ThreadPoolExecutor$Worker@61cb0266 reached by
	reading field java.util.HashMap$Node.key of
		constant java.util.HashMap$Node@21bfcd3e reached by
	indexing into array
		constant java.util.HashMap$Node[]@638fd262 reached by
	reading field java.util.HashMap.table of
		constant java.util.HashMap@4369073a reached by
	reading field java.util.HashSet.map of
		constant java.util.HashSet@599be4fb reached by
	reading field java.util.concurrent.ThreadPoolExecutor.workers of
		constant java.util.concurrent.ThreadPoolExecutor@3ac5047b reached by
	reading field clojure.lang.Agent.soloExecutor

borkdude13:02:17

could also be a call to future

borkdude13:02:22

more likely maybe

borkdude13:02:36

(let [f (binding-conveyor-fn f)
        fut (.submit clojure.lang.Agent/soloExecutor ^Callable f)]

wombawomba13:02:43

...I just realized I'd done a bad copy-paste — rebuilding now with --trace-object-instantiation=java.lang.Thread to see if that helps

wombawomba13:02:06

ah yeah that makes sense

wombawomba13:02:30

I'll try the alter-var-root trick with future as well

borkdude13:02:40

I've certainly come across future calls on the top level which I had to patch. E.g. in reply

wombawomba14:02:08

yeah that's actually it haha

borkdude14:02:09

@wombawomba btw, I think you'd have to patch future-call since that's a function

wombawomba14:02:31

the project has a dependency on reply

wombawomba14:02:52

perhaps I'll just get rid of that for the native-image build

borkdude14:02:54

what kind of project is that then?

wombawomba14:02:07

it's a web service

wombawomba14:02:27

the reply dependency is for debugging/maintenence tasks

borkdude14:02:33

I have a native version of reply, made it for fun, but never released and and I changed the top level future's into delays, worked fine

borkdude14:02:00

but for images it's really the best if I you can get rid of dependencies if you don't need them

wombawomba14:02:58

yeah I'm gonna try that

wombawomba14:02:48

speaking of which, is there a way to exclude requires just for the native-image build?

borkdude14:02:55

you can make the require conditional on some system property.

Property name: "org.graalvm.nativeimage.imagecode"
Values: "buildtime", "runtime"

Property name: "org.graalvm.nativeimage.kind"
Values: "shared", "executable"

borkdude14:02:50

I've also got a related library here: https://github.com/borkdude/dynaload that's designed for optimal graal images.

wombawomba14:02:15

...custom reader conditionals sure would be useful for situations like these

wombawomba14:02:28

I'll take a look at your library, thanks

borkdude14:02:13

you can do this with macros or even just functions in JVM clojure, no need for reader conditionals

wombawomba15:02:16

that library did the trick @borkdude — thanks!

👍 1