Fork me on GitHub
#datalevin
<
2024-06-29
>
Sasha V. Bogdanov01:06:25

Hello! Is it possible to create Linux-arm64 native builds? I badly babashka pod for such architecture.

Huahai02:06:40

You mean native? We have embedded Linux arm64 already. It is possible. Do you know a free CI/CD offering for that platform? Or you can compile your own.

Huahai02:06:09

I mean, free for open source projects

Sasha V. Bogdanov02:06:05

Hmm.. GitHub actions not working with it? Surprisingly.

Sasha V. Bogdanov02:06:20

No, I do not know any. Need to research.

Sasha V. Bogdanov02:06:25

Oh that’s interesting. I will try definitely

Sasha V. Bogdanov02:06:19

But like I see need to make much changes in setup_musl also

Huahai02:06:34

I can give a try when I have time. Right now, I am porting join order benchmarks from SQL. This uncovered several bugs in the query planner. I will release a new version shortly. Then I will try to improve our cardinality estimation further. We are beating Postgres for many queries, but there are a couple horrible cases as well. I will fix those.

🔥 1
Huahai02:06:41

In principle, we should beat Postgres for most cases.

1
Huahai02:06:00

Correct, just add a directory for Linux arm64.

Huahai02:06:16

After we got the planning algorithm nailed, we will do some optimization to reduce planning time. We are doing planning in clojure, so the best we can do is to match the Postgres’s planning time, but we will definitely beat Postgres in execution time for the most part.

👍 2
🚀 1
Sasha V. Bogdanov02:06:10

Thank you for answer. I will try ry on weekend

Huahai04:06:22

The real work of building native is at https://github.com/juji-io/dtlvnative

Huahai04:06:05

Do it there. The main Datalevin project just depends on the artifacts built there.

Huahai04:06:45

All you need to add is a static library build for linux amd64

Huahai04:06:59

Just add a directory like others, and add linux amd64 to the existing .cirrus.yml there

Huahai05:06:43

The static build is through musl tool chain, so you will probably need to hunt for necessary libraries

Huahai05:06:24

libz, libstdc++, etc.

borkdude06:06:13

Amd64? Wasn’t this about arm?

Sasha V. Bogdanov07:06:21

I think it’s typo

borkdude07:06:15

Note that arm64 (yes arm, not amd) doesn't support fully static images, nor musl

borkdude07:06:43

you need to use the "static-except-for-libc" option

Sasha V. Bogdanov07:06:23

Great I will try today. But how will I point it to use my fork of dtlvnative with appropriate changes?

Huahai09:06:01

dtlvnative deploys jars to clojars, you can deploy your fork to your own coordinates, and include in your fork of Datalevin. Look at native/project.clj

👍 1
Huahai09:06:55

after everything tested out, send me PRs to both dtlvnative and Datalevin projects.

Huahai09:06:34

Right now, we are still on an old version of graalvm, as I have not got time to upgrade to the newer version, as it requires code changes on our part, so you will need to use the same version of graalvm, which is 22.3.1

👌 1
Sasha V. Bogdanov12:07:30

I sent both pull requests about ubuntu-aarch64. Please review

Sasha V. Bogdanov13:07:57

Like I understand Windows build must be launched manually? I didn't anything about it

Sasha V. Bogdanov13:07:33

It would be great if you register last version babashka pods for ubuntu-aarch64 as well

Huahai17:07:04

Got them. Thank you! I will do a new release when I am done with the couple of fixes I am working on right now.

1