Hi all, does anyone have a clue why a binary produced by native-image on macOS would try to load libc.so.6 when ran... and not, say, libc.dylib or something similar...? Might someone have ideas on how to fix the below?
Exception in thread "main" java.lang.ExceptionInInitializerError
at java.base@22.0.1/jdk.internal.foreign.SystemLookup.find(SystemLookup.java:43)
at testlib.NativeBridge.ffi(NativeBridge.java:16)
at testlib.core$run_app.invokeStatic(core.clj:11)
at testlib.core$run_app.invoke(core.clj:7)
at testlib.kitchen_sink.main$_main.invokeStatic(main.clj:10)
at testlib.kitchen_sink.main$_main.invoke(main.clj:8)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at testlib.kitchen_sink.main.main(Unknown Source)
at java.base@22.0.1/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
Caused by: java.lang.IllegalArgumentException: Can't load library: libc.so.6
at org.graalvm.nativeimage.foreign/com.oracle.svm.core.foreign.Util_java_lang_foreign_SymbolLookup.createNativeLibraries(Target_java_lang_foreign_SymbolLookup.java:159)
at org.graalvm.nativeimage.foreign/com.oracle.svm.core.foreign.Util_java_lang_foreign_SymbolLookup.libraryLookup(Target_java_lang_foreign_SymbolLookup.java:181)
at org.graalvm.nativeimage.foreign/com.oracle.svm.core.foreign.RuntimeSystemLookup.makeSystemLookup(RuntimeSystemLookup.java:63)
at org.graalvm.nativeimage.foreign/com.oracle.svm.core.foreign.RuntimeSystemLookup.<clinit>(RuntimeSystemLookup.java:39)
... 10 more
Caused by: java.lang.UnsatisfiedLinkError: Can't load library: libc.so.6
at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.NativeLibraries.loadLibraryPlatformSpecific(NativeLibraries.java:90)
at org.graalvm.nativeimage.foreign/com.oracle.svm.core.foreign.Util_java_lang_foreign_SymbolLookup.createNativeLibraries(Target_java_lang_foreign_SymbolLookup.java:156)
... 13 more
The FFI code failing is merely this:
package testlib;
import java.lang.foreign.Arena;
import java.lang.foreign.FunctionDescriptor;
import java.lang.foreign.Linker;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.SymbolLookup;
import java.lang.foreign.ValueLayout;
import java.lang.invoke.MethodHandle;
public class NativeBridge {
public static long ffi() throws Throwable {
Linker linker = Linker.nativeLinker();
SymbolLookup stdlib = linker.defaultLookup();
MethodHandle strlen = linker.downcallHandle(
stdlib.find("strlen").orElseThrow(),
FunctionDescriptor.of(ValueLayout.JAVA_LONG, ValueLayout.ADDRESS)
);
try (Arena arena = Arena.ofConfined()) {
MemorySegment cString = arena.allocateFrom("Hello");
long len = (long)strlen.invokeExact(cString); // 5
return len;
}
}
}
And the native-image command to produce the binary was this:
> $GRAALVM_HOME/bin/native-image \
> -jar target/testlib-kitchen-sink-0.0.1-standalone.jar \
> -cp target/testlib-kitchen-sink-0.0.1-standalone.jar \
> -H:+ReportExceptionStackTraces \
> -H:+UnlockExperimentalVMOptions \
> -H:+ForeignAPISupport \
> -J-Dclojure.spec.skip.macros=true \
> -J-Dclojure.compiler.direct-linking=true \
> --initialize-at-build-time \
> --no-fallback \
> --report-unsupported-elements-at-runtime \
> --native-image-info \
> --enable-url-protocols=http,https \
> --verbose \
> -march=native \
> target/testlib-kitchen-sink-0.0.1-standalone
Environment:
❯ java -version
openjdk version "22.0.1" 2024-04-16
OpenJDK Runtime Environment Zulu22.30+13-CA (build 22.0.1+8)
OpenJDK 64-Bit Server VM Zulu22.30+13-CA (build 22.0.1+8, mixed mode, sharing)
❯ $GRAALVM_HOME/bin/native-image --version
native-image 22.0.1 2024-04-16
GraalVM Runtime Environment Oracle GraalVM 22.0.1+8.1 (build 22.0.1+8-jvmci-b01)
Substrate VM Oracle GraalVM 22.0.1+8.1 (build 22.0.1+8, serial gc, compressed references)
The FFI call runs fine when the jar is ran with java, but it fails when ran from the binary produced by native-image.Are you running the binary on the same system as where you compiled it?
Yep.
Someone has just replied to the same question in the graalvm slack. It seems macos is not really supported for FFI yet: https://github.com/oracle/graal/blob/81d23df042d19a359b3a978f776744fa310bf93a/substratevm/src/com.oracle.svm.core.foreign/src/com/oracle/svm/core/foreign/RuntimeSystemLookup.java#L60-L66
what is the output for otool -L ./the-binary?
oh ok
Is this the new Panama stuff?
You can do FFI via the bespoke graalvm C API https://yyhh.org/blog/2021/02/writing-c-code-in-javaclojure-graalvm-specific-programming/
or JNI
but it would be nice if Panama would be supported in the future
I've got a Rust example here: https://github.com/borkdude/clojure-rust-graalvm
I have the jni version working... and was merely looking into whether I could avoid writing all the native bridge code. O:-)
A constraint I have on all of this though, making a native binary have to be optional in my case. Will the graalvm C API approach work when the code utilising it runs in a normal jvm?
No, that only works in a native-image so you have to write that FFI code twice in that case, unless you use JNI (or JNA)
I think datalevin (project by the same author as the article I linked) did this too I believe
Thanks heaps for the article link. I'll dig a little more into this... might as well just resort to using ffi anyway and maybe, by the time I'm finished with the implementation, graalvm would have landed a proper Panama support. 🤞
There are a couple of options available that are in flux while Panama becomes supported more widely. You can use JNA, which can be used on the JVM and when compiled via native image. It's slower compared to JNI and Panama, which may or may not be acceptable depending on your use case.
dtype.next has a generic ffi interface with implementations for JNA and graalvm, https://cnuernber.github.io/dtype-next/tech.v3.datatype.ffi.html. I think there is some support for an old version of Panama.
Whoa, I guess I was mistakingly under impression JNA wasn't going to work in native image after reading https://github.com/oracle/graal/issues/673, which said "Full support for JNA is still not available or planned". 😁 But after reading your suggestion, I've decided to give it a try, and as far as I'm concerned it seems to work just fine. So I guess it might be the best option for me for now. Thanks heaps!