It would be nicer to bundle the native lib in the jar, as most JVM libraries using C libraries do

@U0A74MRCJ, I mostly agree. Do you know any good tools for building and deploying jars with native libs?

It’s a matter of putting them in the resource directory of the jar.

No extra tools necessary, as far as I can tell.

Right, but you also have to build the library for multiple architectures and platforms, right?

Right, that’s unavoidable. We all have to do that.

Is there an easy way to do that? My experience is that it's a pain.

I use a bunch of empty lein projects to do it, you may use clj build, or maven, or whatever

Depends on what build tool you are using already

Do you cross compile? or use some cloud CI service for the various platforms?

I use cloud CI

That’s the price to pay for using native library. It’s not a big deal. Once it’s setup, it’s not a problem.

You can’t expect your JVM users to do these themselves though, unless you don’t want people to use it.

That's what package managers are for.

I do agree that jar dependencies are easier though.

I don’t think package managers can do this for you, as native libraries are different, and platforms are numerous

Besides, you want to have control of the stack because your users will come to you

I tested clj-graphviz on mac and linux using the libs built by the system package managers.

You can’t expect users to install this and that, because your users are JVM users, they use Java to avoid the native dependency hell.

If possible, build your native libs statically even.

Can you load static libraries from the JVM?

I thought they had to be shared/dynamic libraries.

I think so

https://docs.oracle.com/javase/8/docs/api/java/lang/System.html#loadLibrary-java.lang.String

This only works for GraalVM though

For regular JVM, I don’t think so, as you are not going to build the VM.

On GraalVM, static link only works for amd64 linux at the moment

But that’s what we are doing.

It’s a lot of work, for musl tool chain has to be used for static linking.

Anyways, as a library maintainer, that’s the work you have to do, because the users will come to you. You are responsible for the whole stack, from bottom to top. LOL

hmmm, I wonder if there's a way to automatically package native dependencies by just running the package manager script and putting that in a jar?

What package manager you are referring to? Apt? yum? They are providing for the OS. Putting the deps in the jar is the opposite of that: you are not using OS desp, you are providing deps of your own.

The reason to use JVM is because it is standalone, not OS dependent. That’s the whole Java story. Your users expect that story to stick, even your library depends on native things. That’s why we bundle .so, .dll, etc. in the jar.

Trying to use OS lib is more hustle than just compile your own and put them in the jar.

It’s much less work, trust me.

I do this work for my other library membrane, and it's a pain.

If there's an easier way, I'm definitely open to it. I also have wrappers for other c libraries like ffmpeg, chromium embedded framework, and libclang. It would be great to have an easy way to provide native deps for all these.

You are welcome to come up with something. Would love to see it.

Do you have any open source examples that show how easy it is?

This is what I do for datalevin: https://github.com/juji-io/dtlvnative

just copy paste a bunch of files around

Thanks, I'll check it out.

Maybe it's an idea to decouple the native lib from the clojure code, as the underlying stuff (like compiled Java) often doesn't change as much and then you won't have to upload/download these binaries for every minor clojure change

Yea, it’s pretty common to put the native lib in its own jar. It’s also common to use a different jar for each target platform and architecture so you can download only the deps that are actually required.

Neat work @U7RJTCH6J ps maybe old news for you, but have you tried https://openjdk.org/jeps/434? (I've used it with some success myself...)

Not directly, but I have used https://github.com/IGJoshua/coffi which wraps it.

The c wrapper firsts generates a data representation of the c API by parsing the headers and then generates JNA code that does all the ffi. I would also like to have a code generator that uses panama. I really like the #coffi API, but there's some missing pieces for using structs by reference without fully serializing all the data (which I usually rely on).