Fork me on GitHub
#xtdb
<
2022-05-21
>
nivekuil03:05:41

trying to build an xtdb native image, but it seems reflection needs to be accounted for. https://github.com/xtdb/xtdb/blob/master/dev/build-graal-native-image.sh references a graal_reflectconfig.json, is that available anywhere?

refset09:05:23

I can dig through the archives next week, but it's likely to be well out of date. Nobody at our end has run the script in a very long time and Graal has been a moving target since I believe 🙂

nivekuil11:05:23

that's a shame, I think native image would be very helpful for a hosted XTDB. I think it should be doable, but I don't want to be the one to facecheck it!

refset13:05:34

Our testing at the time suggested that the native image build could only really be useful for faster cold-start, time-to-first-query usage, and overall performance/throughput was a bit lower ...which is roughly what I had expected based on everything I'd read, but maybe the situation is different now(?)

refset13:05:15

A quick Google suggests that the reflectconfig file is generated by the native-image-agent - have you tried that?

nivekuil14:05:22

I did find that, currently stuck with

Fatal error: com.oracle.svm.core.util.VMError$HostedError: com.oracle.svm.core.util.VMError$HostedError: Registering type as reachable after analysis: AnalysisType<java.nio.file.WatchEvent$Kind[], allocated: false, inHeap: false, reachable: false>  at com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:72)  at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:678)  at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:515)  at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:407)  at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:585)  at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:128)  at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:615)

nivekuil14:05:37

the main benefits are much less RSS and smaller container image

nivekuil14:05:25

if we're handing out free XTDB instances, probably couldn't afford more than 128-256mb so that becomes critical

refset14:05:03

That's a bummer! Can't help you there :/ And yep, hosting efficiency at small scale is a valid use case too. Good point 👍

refset14:05:17

XTaaS definitely isn't out of the question for us at some point 🙂

👍 2
nivekuil23:05:32

hah, it may be closer on my roadmap than yours! I'll look at datalevin for now since it already supports native image

refset10:05:05

Well...now I'm very intrigued!

refset10:05:18

Out of interest, were you hoping to stick with xtdb-lmdb? Or would you make do if only xtdb-rocksdb was supported with native image?

nivekuil11:05:55

just rocksdb -- I'm having a lot of fun with my infra framework, which has exceeded all my expectations so I'm looking to flex my muscles a bit 🙂

🌞 1
Benjamin C05:06:41

I am also interested in this for the small container size and low memory use-case. If y'all are able to get something working (I don't particularly care which backend) I'd be happy to hear about it. 🙂

📝 1
👍 1
mattias06:01:02

Has anyone tried GraalVM native image build with in-memory XTDB, either on v1 or v2? Using graalvm 17, my native-build gets to a point where org.agrona.concurrent.UnsafeBuffer.byteBuffer causes issues, which is a dependency of xtdb v1. If there is hope with v2, I might try to upgrade and retry the build. I use native-build so that I can conveniently package my app, which (among other things) includes an in-memory xtdb. I suppose I could use some in-memory DB to begin with, but I'd like to keep my options open in case I need to move the DB out.

mattias06:01:02

Has anyone tried GraalVM native image build with in-memory XTDB, either on v1 or v2? Using graalvm 17, my native-build gets to a point where org.agrona.concurrent.UnsafeBuffer.byteBuffer causes issues, which is a dependency of xtdb v1. If there is hope with v2, I might try to upgrade and retry the build. I use native-build so that I can conveniently package my app, which (among other things) includes an in-memory xtdb. I suppose I could use some in-memory DB to begin with, but I'd like to keep my options open in case I need to move the DB out.