Fork me on GitHub
#xtdb
<
2022-12-23
>
Mark Wardle20:12:15

Hi all, I am continuing to experiment with xtdb but have a question about lmdb and why xtdb pulls in both lmdb-java and the lwjgl libraries, and how you manage to get this working on ARM.... [more detail in thread]...

Mark Wardle20:12:08

I currently use lmdb-java in another project, used as a library, and so have to use an x86 jvm on my apple silicon mac (M1 pro) as there are no ARM binaries [https://github.com/lmdbjava/native/issues/10 - although perhaps fixable]. Works fine using x86. I cannot run xtdb with lmdb except when using an ARM based jdk (I get a weird SIGABRT with 'loading libcrypto in an unsafe way'). Works fine on ARM. This leaves me in an awkward place, as either I switch to develop on linux, see if I can manually compile lmdb on ARM and get it working for lmdb-java, or look at how xtdb is managing to use lmdb on ARM. I see from the xtdb-lmdb module that you depend on lmdb-java and lots of lwjgl versions (https://github.com/xtdb/xtdb/blob/master/modules/lmdb/project.clj). My assumption then was you were using lmdb-java API and using the compiled native binaries from lwjgl, but then the source code clearly uses lwjgl-lmdb and I couldn't see any use of lmdb-java on cursory inspection of the code.

Mark Wardle20:12:41

Would be keen to understand the trade-offs between lmdb-java and this lwjgl library, and whether xtdb/lmdb can work on both ARM and x86 without those libcrypto errors.

wotbrew10:12:11

Hi Mark, I'll take a look at the dep on lmdb-java, you are right, I suspect it is unused. Here is the commit that introduced M1 support https://github.com/xtdb/xtdb/commit/ee70e46060446f9909d53d2898585c4609d30386, looks like I bumped the lwjgl deps. As to why lwjgl, unfortunately I am not yet the best person to ask re trade-offs with lmdb-java, but I too am curious so I'll try and find out for you.

Mark Wardle11:12:44

Thanks @U0GE2S1NH - I think lwjgl obviously provides the native libraries for the different architectures, and I couldn't see that you are using lmdb-java at all, but I think it pulls in native libraries too, so there may be unintended consequences in removing it, but I don't know. As it is, as I'm using the API provided by lmdb-java and not lwjgl. When I was using an x86 JDK, I had problems with xtdb and libcrypto, but I managed to compile a native aarch64 library and switch away from an x86 JDK, so I'm now running natively and without difficulty with both lmdb-java and xtdb in the same classpath.

👍 1