Fork me on GitHub
#graalvm
<
2020-03-09
>
borkdude12:03:49

pgmig was updated from clojure 1.8.0 (!) to clojure 1.10.2-alpha1: https://github.com/leafclick/pgmig/commit/68864abe8e94910402e6a01d7de88f5d0cdfc504

katox13:03:04

@mkvlr I have a frontend rest service running in native image:

[tiketguard:26315]    classlist:  10,791.61 ms
[tiketguard:26315]        (cap):   1,308.11 ms
[tiketguard:26315]        setup:   3,329.41 ms
[tiketguard:26315]   (typeflow): 323,689.19 ms
[tiketguard:26315]    (objects):  78,563.42 ms
[tiketguard:26315]   (features):  10,511.45 ms
[tiketguard:26315]     analysis: 419,706.99 ms
[tiketguard:26315]     (clinit):   1,737.86 ms
[tiketguard:26315]     universe:   6,046.41 ms
[tiketguard:26315]      (parse):  11,295.08 ms
[tiketguard:26315]     (inline):  13,751.14 ms
[tiketguard:26315]    (compile):  65,125.46 ms
[tiketguard:26315]      compile:  95,394.01 ms
[tiketguard:26315]        image:  14,677.93 ms
[tiketguard:26315]        write:   1,736.10 ms
[tiketguard:26315]      [total]: 552,193.88 ms

katox13:03:43

it seems to work normally with 1.10.2-alpha1 as well

katox13:03:45

hmm, no premature statement

katox13:03:10

it compiles though 🙂

katox13:03:01

I think it culprint is some of the deps having spec in its deps (and not in the latest version)

borkdude13:03:28

shouldn't 1.10.2-alpha1 pull in the latest spec, or does it get overriden somehow?

borkdude13:03:44

clj -Spath or lein classpath should tell you the answer

borkdude13:03:13

could also be that one dep of yours is AOT-ed with clojure 1.10.1 or so

katox13:03:20

spec.alpha-0.2.187.jar

borkdude13:03:31

what error are you seeing?

katox13:03:46

I'll have to investigate. But the message is what I see when "wrong" spec is in com.oracle.svm.core.jdk.UnsupportedFeatureError: Invoke with MethodHandle argument could not be reduced to at most a single call: java.lang.invoke.Invokers$Holder.invoke_MT(Object, Object, Object, Object)

borkdude13:03:56

@katox that's a JDK11 related thing.

katox13:03:10

there are deps that don't include stuff when using clj 1.8

borkdude13:03:27

graalvm-xxx-java8

katox13:03:44

yet pgmig works with it

borkdude13:03:07

yeah, it depends whether your program will hit that piece of code I think

katox13:03:40

It seems so. Bunch of code is spec related and thus not included when using clj 1.8.

katox13:03:19

That'd make sense. I tried to avoid reflection calls in pgmig but that's not feasible in the larger app. I'll try to play again with it soon

borkdude13:03:47

the patch works well

katox13:03:05

for now I'll try again with graal on java8

katox13:03:14

@borkdude good tip, the reflector error is gone on java8 graal. There are still problems in the final binary but the build is on track now.

borkdude15:03:26

I think I found a better fix for the clojure.lang.Reflector issue with java11! https://github.com/oracle/graal/issues/2214#issuecomment-596598421 No need anymore to patch Clojure itself by overriding the class.

katox15:03:57

I updated some more deps and now the full app works with java8 graal!

borkdude15:03:34

I'll make some proper docs around this java11 fix later

katox15:03:30

@borkdude is the above a configuration class that can be included in the project along with json configuration? Because I tried the java configuration code before and ran into more troubles.

borkdude15:03:32

@katox This is the fix in action: https://github.com/borkdude/sci/commit/4c8190c0e95860ece8e8b195b05d31ad0866d763 you need to compile this with svm.jar on the classpath. The commit shows that.

😮 4
Crispin15:03:58

Very impressive @borkdude.

Crispin15:03:59

So am I reading this right. This works both graal java 8 and java 11. And keeps full functionality in both.

borkdude15:03:27

I think I could even make a library out of this

borkdude15:03:52

I'll try this later this week

Crispin15:03:55

You have been busy! How did you learn about @TargetElement? Just reading source code?

borkdude15:03:27

it came up in another issue. See https://github.com/oracle/graal/issues/2136 towards the end.

Crispin15:03:43

Ah yeah, I see. Very nice. This is so much better.

lread15:03:44

Oh that’s interesting @borkdude, so with Java substitutions, we can monkey patch Clojure!

borkdude15:03:02

yes, at least the Java bits

borkdude15:03:27

so if Clojure is going to have more Java-version specific bits, we can maybe work around it this way

lread15:03:14

yeah… neat! Another tool in the chest!

katox15:03:31

I think oracle provided some maven svm artifact. You could put it as a native image profile dependence.

borkdude15:03:20

ah nice. I'll use that then

borkdude15:03:03

it is GPL licenced. is this what the classpath exception is for, i.e. when I compile my binary with this, I can still distribute it?

borkdude15:03:20

This is from the native-image chat:

Michiel Borkent  [4:22 PM]
Is the svm.jar available as a maven library so I can put it on my classpath via a package manager?


Michiel Borkent  [4:38 PM]
it seems it is, but not updated for newer versions?  (edited) 

David Lloyd  [4:49 PM]
the group ID changed to org.graalvm.nativeimage
[4:49 PM]


The Central Repository Search Engine
The Search Engine for The Central Repository

Michiel Borkent  [4:50 PM]
oh thank you!

Peter Niederwieser  [4:50 PM]
afaik the jar can't currently be used with openjdk.

katox15:03:28

But you don't use it with openjdk. You use graal and the graal's native image to create and link the binary. And the binary on linux is linked to some gnu libc stuff.

Crispin16:03:23

glibc (just that lib) is also LGPL not GPL.

Crispin16:03:59

the gnu library that normally causes linking license problems is libreadline (its GPL)

katox16:03:05

You're right. Still there is no openjdk involved unless you use that on CI server.

borkdude16:03:30

I'll process this fix into proper docs and maybe a lib later this week(end)

katox17:03:42

Just FYI native-image compilation times for the FE service app - various -J-Xmx settings

16G
[tiketguard:21475]      [total]: 280,823.65 ms
10G
[tiketguard:30357]      [total]: 874,997.50 ms
8G
java.lang.OutOfMemoryError

jumar21:03:06

Interesting speedup going from 10 to 16 G

katox17:03:30

for reference the uberjar size is about 40megs (got a heavy cut to actually compile and run as a native-image)

borkdude19:03:41

Feedback welcome. If it also works for you, I'll drop the alpha label

borkdude19:03:07

It worked for sci.