Fork me on GitHub
#graalvm
<
2021-07-23
>
Karol Wรณjcik18:07:54

Btw has anyone benchmarked the performance changes in 21.2?

ericdallo16:07:22

I'm interested on that as well

borkdude18:07:14

@ericdallo I've helped @chandru89new by making sqlite compile in a native-image using your "fix" library. https://github.com/borkdude/clj-sqlite-graalvm-native I had to change a couple of clojure-lsp specific config things. But after compiling I still get:

$ ./tt foo
Exception in thread "main" java.lang.UnsatisfiedLinkError: org.sqlite.core.NativeDB._open_utf8([BI)V [symbol: Java_org_sqlite_core_NativeDB__1open_1utf8 or Java_org_sqlite_core_NativeDB__1open_1utf8___3BI]
	at com.oracle.svm.jni.access.JNINativeLinkage.getOrFindEntryPoint(JNINativeLinkage.java:153)
	at com.oracle.svm.jni.JNIGeneratedMethodSupport.nativeCallAddress(JNIGeneratedMethodSupport.java:57)
	at org.sqlite.core.NativeDB._open_utf8(NativeDB.java)
	at org.sqlite.core.NativeDB._open(NativeDB.java:78)
	at org.sqlite.core.DB.open(DB.java:195)
	at org.sqlite.SQLiteConnection.open(SQLiteConnection.jav
...

borkdude18:07:21

Perhaps you can take a look at this.

borkdude18:07:03

You can run this project by running ./compile

ericdallo18:07:10

Sure, Probably something I forgot to migrate from clojure-lsp to the lib

ericdallo18:07:23

I can try that tomorrow

๐Ÿ‘ 3
๐Ÿ™ 3
borkdude18:07:35

Perhaps it's just missing the dynamic lib inside the generated image

borkdude18:07:45

@ericdallo @chandru89new found the issue, PR coming

borkdude18:07:10

the issue is that you should not do any queries on the top-level in a namespace, since those run at build time and this causes strange problems.

โ˜๏ธ 3
๐Ÿ‘ 3
Chandru18:07:23

running the new code now

ericdallo18:07:04

Good catch! Also, I remebered I created this sample project using that fix lib: https://github.com/ericdallo/sqlite-graalvm-sample

๐Ÿ‘ 3
borkdude18:07:08

I still think it's better to generate some JSON files and put that into resources/META-INF of your fix library @ericdallo, that works equally well and doesn't give these incompatibilities with graalvm versions

ericdallo18:07:23

I tried to do that one time for clojure-lsp to avoid using that lib but it didn't work ๐Ÿ˜… I probably missed something

borkdude18:07:43

you can still have that lib, but that lib can just expose those JSON files

borkdude18:07:47

instead of those Java classes

ericdallo18:07:07

yes, sounds way better

borkdude18:07:14

perhaps the xerial library will even accept a PR for it

ericdallo18:07:58

that would be the best IMO

ericdallo18:07:08

I'll can try that this weekend

Chandru18:07:19

So a question. How is it that Ericโ€™s sample (https://github.com/ericdallo/sqlite-graalvm-sample) works without any of those META-INF files?

borkdude18:07:58

The registration of the sqlite classes happens in his "fix" lib

๐Ÿ˜ฎ 3
ericdallo18:07:59

Because I didn't know that you can just add the configs via META-INF resources ๐Ÿ˜…

ericdallo18:07:45

so part of the config comes from the fix lib and other is manually added via this script

Chandru18:07:41

@borkdude your PR#2 fixed it and things are working great now! Cant tell you how thankful I am to you, @ericdallo and @karol.wojcik for all the help.

๐ŸŽ‰ 6
borkdude18:07:39

I should perhaps add sqlite to https://github.com/babashka/babashka-sql-pods as well :)

ericdallo18:07:51

@borkdude it still doesn't work as static though ๐Ÿ˜•

Chandru19:07:03

i feel embarrassed to ask this but my excuse is Iโ€™m mostly a Javascript/Typescript guy, but what does this actually mean in this context of babashka pods?

borkdude18:07:06

oh crap yeah, static for pod is definitely nice to have

borkdude18:07:34

this one does work with bb already https://github.com/babashka/pod-babashka-go-sqlite3

๐Ÿ‘ 3
borkdude18:07:58

@ericdallo it seems like forking xerial yourself to make it static might be better, then try it out a while and then PR the changes back?

ericdallo18:07:36

yes I just have no idea how to change those makefile/gcc args to work ๐Ÿ˜‚

ericdallo18:07:02

It seems the best would be to change gcc to use musl to be real static

borkdude18:07:38

yes, but then that lib doesn't work with non-gcc environments. I'm not sure if what you want is possible at all

๐Ÿ˜” 3
borkdude19:07:28

this is why using a pure Java library has some benefits here (h2, hsqldb), none of these problems

ericdallo19:07:50

yes ๐Ÿ˜•

borkdude19:07:12

somebody should do a pure Java implementation of sqlite :)

ericdallo19:07:46

Yeah, I can't believe we don't have that, android is made in Java and use that all the time haha

borkdude19:07:57

they also use the C one

borkdude19:07:02

there is only one I think, in C

ericdallo19:07:17

it makes sense

borkdude19:07:35

perhaps clojure-lsp can move to hsqldb or h2?

borkdude19:07:45

why is it so attached to sqlite?

ericdallo19:07:41

last time I talked with @snoe, he wanted to make sure transactions were atomic

borkdude19:07:43

btw, the xerial one, we could also port that to use the GraalVM C API, then it would work statically I think

ericdallo19:07:02

how hard is to do that?

Chandru19:07:03

i feel embarrassed to ask this but my excuse is Iโ€™m mostly a Javascript/Typescript guy, but what does this actually mean in this context of babashka pods?

borkdude19:07:22

> how hard is to do that? it's probably just a lot of work ;)

๐Ÿ˜† 3
borkdude19:07:35

@chandru89new do you know babashka?

Chandru19:07:51

Not really, @borkdude. In my quest to find a way to write clojure and use sqlite for this small project, I did stumble on your repo and precisely the sql-pods repo and had to back out coz I didnt find sqlite in the list.

borkdude19:07:28

ok, babashka is a clojure scripting environment that runs in a native binary with instant startup

borkdude19:07:02

it doesn't have any sql drivers but this is where pods come in: there are pods for every major db and you can still have access to those dbs via the pods

borkdude19:07:21

but it's nice to have pods to be statically compiled so they work in "every" linux: alpine, nixos and ubuntu

๐Ÿ‘ 3
Chandru19:07:13

Oh wow. Would this be like some super-charged version of doing node index.js except that I dont need the environment or anything because bb comes with the SCI? Plus, faster?

borkdude19:07:53

bb supports a large subset of Clojure but not everything

borkdude19:07:57

and it can't load external Java classes

borkdude19:07:04

but it contains the most commons ones.

borkdude19:07:25

so you can do most things that you will need for scripting, but there are some limitations

Chandru19:07:59

Understood! There are so many interesting things to learn and try ๐Ÿ˜„

borkdude19:07:53

You can see babashka as a replacement for bash scripting using Clojure instead

๐Ÿ‘ 3
borkdude19:07:02

(but it can do a bit more than that, which is a bonus)

ericdallo19:07:16

if not, you should really take a look, it's amazing ๐Ÿ˜‰