Clojurians Log v2
graalvm
Hi folks!
I'm trying to experiment a thing here with native-image - I want to expose an API from Clojure to C. So far, I got almost everything I needed, but - there's a Clojure object I need to pass as a parameter. Is there a way to wrap arbitrary, opaque Java or Clojure objects?
Essentially, what i want to do is to make void* some_state = call_some_java_function(....); in C, and pass this some_state to some other functions
I'm not sure about this but this article may contain some pointers: https://yyhh.org/blog/2021/02/writing-c-code-in-javaclojure-graalvm-specific-programming/ The author is on this slack as well
Well...... I might need some more help, actually. It seems that this post is showing how to make C structures visible on Java side, and I want the opposite - to make a Clojure data visible on C (I don't actually need the C code to do anything with it - just need to be able to pass around functions). It's amazing how little documentation I can find on this...
I think you need to pass posters around and they also do something named pinning to prevent GC
Posters = pointers
There's more than one way to do it. You can have call_some_java_function return an int handle and keep a mapping of int-handle->obj which can translate the opaque type to the internal type at the boundary.
From https://www.graalvm.org/sdk/javadoc/org/graalvm/nativeimage/c/function/CEntryPoint.html > No object types are permitted for parameters or return types; only primitive Java values, word values, and enum values are allowed. Enum values are automatically converted from integer constants to Java enum object constants. You could probably cook something up using JNI, but it's pretty straightforward to just keep a mapping of handles.
that id -> object is exactly what I'm doing in babashka pods as well for objects that you can't serialize across the boundary
but I would expect a better solution in the native world
I think the problem with trying to get a pointer to any random data is that it would interfere with the garbage collector and/or optimizations. For it to work, you would end up having to create a similar mapping anyway (see https://www.graalvm.org/22.0/reference-manual/native-image/JNI/#object-handles).
the interference with gc is done using pinning, right? https://www.graalvm.org/sdk/javadoc/org/graalvm/nativeimage/PinnedObject.html
Yea, hadn't seen that. That could work. You would still have to manage the lifetime of the object if you pass the pointer into native code.
How did I not see that PinnedObject? Thanks, will try with that
In any case... I made it work without using PinnedObject somehow 😄
Are you sure you want native image for this?
it sounds like, if youre just passing clojure data around C, that the FFM api would be potentially useful
I want to call the Clojure code from other languages, like Ruby - Ruby being the "main language" in this case. Even if I had some way to call Clojure (or any JVM language) from Ruby (or C, in this case), it's a hard sell to say to someone "install a JVM, and this jar, and then you can use this Ruby library..."
Gotcha, so not JRuby
Or graals ruby
No JRuby indeed. It's hard, close to impossible, to find a Ruby codebase that expects (or even supports) alternative implementations.
Well, another batch of errors - I am trying to load Pathom in my shared library, but it's failing with "Detected a started Thread in the image heap.". Tracing the initialization seems to point to Guardrails loading core.async's "thread_pool_executor":
at java.lang.Thread.<init>(Thread.java:493)
at clojure.core.async.impl.concurrent$counted_thread_factory$reify__840.newThread(concurrent.clj:28)
at java.util.concurrent.ThreadPoolExecutor$Worker.<init>(ThreadPoolExecutor.java:630)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:920)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1353)
at clojure.core.async.impl.exec.threadpool$thread_pool_executor$reify__854.exec(threadpool.clj:32)
at clojure.core.async.impl.dispatch$run.invokeStatic(dispatch.clj:35)
at clojure.core.async.impl.dispatch$run.invoke(dispatch.clj:32)
at com.fulcrologic.guardrails.core$fn__9570.invokeStatic(core.cljc:63)
at com.fulcrologic.guardrails.core$fn__9570.invoke(core.cljc:62)
at com.fulcrologic.guardrails.core__init.load(Unknown Source)
at com.fulcrologic.guardrails.core__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Unknown Source)
at java.lang.Class.forName(Class.java:467)
Yes! Basically replace the real implementation of guardrails with noop namespace.
Did you also try to disable the guardrails via the property?
No, I upgraded guardrails and that made the problem go away 😅