Fork me on GitHub
#graalvm
<
2020-10-01
>
sergey17:10:44

Hey y'all - a while back I posted in here asking about using the cognitect AWS api from native-image. I wanted to share that I figured out how to do it and https://github.com/latacora/native-cogaws that shows how to get s3 read and write calls working

👍 6
sergey17:10:22

The key insights: 1. you need to provide configuration files for Reflection, resource loads, etc. 2. the cognitect aws client should not be instantiated in a global def (put it in a function or wrap it in a delay) 3. you need to require two internal https://github.com/latacora/native-cogaws/blob/master/src/latacora/native_cogaws.clj#L4 in the file that uses the client

sergey17:10:21

#3 was something that was suggested in this channel, so thanks for the tip (I think it was from @borkdude or @U0FT7SRLP?)

sergey17:10:05

One thing I'm curious about: why is it that requiring those namespaces helps resolve this runtime error:

Exception in thread "main" Syntax error compiling fn* at (cognitect/aws/http/cognitect.clj:1:1).
... (stack trace)
Caused by: com.oracle.svm.core.jdk.UnsupportedFeatureError: Unsupported method java.lang.ClassLoader.defineClass1(ClassLoader, String, byte[], int, int, ProtectionDomain, String) is reachable

borkdude17:10:36

GraalVM can't create load classes at runtime, so that might be the issue

borkdude17:10:07

How big is the binary that is resulting from the reflection based approach?

borkdude17:10:17

I figure it would be quite big quite fast?

borkdude18:10:05

And what memory usage is reported during compilation?

sergey19:10:00

compilation memory usage goes up to 7.2GB; the binary size is around 82MB (for native-cogaws)

borkdude19:10:50

yeah, that's what I was afraid of

borkdude19:10:02

how long does compilation take?

borkdude19:10:23

on Github actions? or ?

sergey19:10:35

just locally, I haven't tried via GH actions

borkdude19:10:52

What CPU do you have?

sergey19:10:47

Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz (from /proc/cpuinfo)

borkdude19:10:37

ok thanks. This is similar what I got with hato, memory spikes. I think this can be heavily optimized for native.

borkdude19:10:02

Usually runtime require/find-var/resolve etc spike memory + binary size

sergey19:10:28

How would you go about optimizing? All I could think of was removing entries from config files, but that seems pretty time consuming/prone to bugs

borkdude19:10:40

I don't know the cognitect library very well, but I think the approach taken by the pod-tzzh-aws library is more promising: instead of a runtime approach a compile time approach could work better, where all code paths are generated before hand and no reflection / dynamic requires are done at runtime.

sergey19:10:46

> where all code paths are generated before hand and no reflection / dynamic requires are done at runtime Is this different from what happens when you feed native-image the reflection config file?

borkdude19:10:52

Not sure, but for what it does (a couple of calls to s3) I think 7-8GB and 80mb is way too much

borkdude19:10:39

I think @U0FT7SRLP also looked into this once, maybe he can tell you more

sergey19:10:36

yea, seems pretty excessive. We had another cli app (just s3 calls) that went up from 60MB to 100MB after I added the reflection config. I haven't tried on more involved apps, so can't say if more services will mean more CPU/binary size

borkdude19:10:46

If I was a heavy user of cognitect aws or AWS in general, which I am not at the moment, I would probably rewrite that thing for GraalVM specifically

jeroenvandijk20:10:44

@sergey923 You might have seen my fork of cognitect's library https://github.com/AdGoji/aws-api It solves the case where you need to do something specific and you want to take that extra effort to have a standalone binary (no python or other dep). For quick ad hoc AWS calls it's not suited, but it takes the signing of requests (hardest part). I'm guessing it would be possible to run the whole library with graalvm if you get rid of some of the lazy loading (in the http client and credential lookup mostly I think). There is also some XML parsing that might bloat the binary more

sergey17:10:05

One thing I'm curious about: why is it that requiring those namespaces helps resolve this runtime error:

Exception in thread "main" Syntax error compiling fn* at (cognitect/aws/http/cognitect.clj:1:1).
... (stack trace)
Caused by: com.oracle.svm.core.jdk.UnsupportedFeatureError: Unsupported method java.lang.ClassLoader.defineClass1(ClassLoader, String, byte[], int, int, ProtectionDomain, String) is reachable

ghadi17:10:47

the cognitect aws-api dynamically loads namespaces based on the target service's protocol

ghadi17:10:51

s3's protocol is rest-xml

ghadi17:10:02

sqs's protocol is query

borkdude18:10:58

Side note: this is a way to call AWS at native (startup) speeds using babashka: https://github.com/tzzh/pod-tzzh-aws It's a pod leveraging the Go API but the Go code is generated using babashka itself (could also be done using Clojure, detail)

borkdude18:10:12

I guess it's a similar approach as cognitect AWS API except the code generation as all done beforehand, no reflection stuff

borkdude18:10:17

(he also published this today: https://github.com/tzzh/pod-tzzh-mail - similar approach leveraging a Go mail library)

borkdude18:10:24

Babashka pods can also be called from the JVM btw

ghadi19:10:43

@borkdude aws-api does not do reflection

ghadi19:10:11

it reads edn descriptor files that describe the service endpoints

borkdude19:10:28

Ah, thanks. There may be other issues why it's heavy on GraalVM native-image. One issue at a glance: $ ls src/cognitect/aws, I spot that it has a dynaload namespace. If that's similar to what spec does, it already explains something. I made a variant of dynaload with specific GraalVM settings: https://github.com/borkdude/dynaload

ghadi19:10:09

for AOT, you can preload all the dynaloaded things, like the repo does above

ghadi19:10:39

the set of dynaloaded namespaces is bounded

borkdude20:10:19

yes, but even then, having code around with find-var, require on the non-top-level (function bodies) in it can still make the image more bloated than necessary. It will work, but it will also be more bloated than necessary. See e.g. https://clojure.atlassian.net/browse/CLJ-2582

borkdude20:10:25

What I would probably do is fork that lib, get rid of these references and optimize

borkdude20:10:04

If it doesn't do reflection, it is weird why such a huge reflection config is necessary in the AOT-ed lib to make it work

ghadi20:10:50

I don't know, I haven't analyzed it. Speaking with my maintainer hat on, aws-api does not explicitly do anything "reflective". It just delays the loading of a few namespaces.

ghadi20:10:02

needs core.async, xml, json libraries, + jetty

borkdude20:10:40

Maybe it would already help if this part was avoided:

src/cognitect/aws/dynaload.clj
14:    (or (resolve s)

ghadi20:10:44

reflect-config.json seems to include cheshire, which we don't use

sergey20:10:07

that might have been me experimenting with some stuff

borkdude20:10:14

yeah, I saw that, seems weird

ghadi20:10:11

the only truly dynamic thing we plan on doing is having a pluggable http client

ghadi20:10:46

right now it's hardcoded to jetty

borkdude20:10:04

is it?

(defn resolve-http-client
  [http-client-or-sym]
  (let [c (or (when (symbol? http-client-or-sym)
                (let [ctor @(dynaload/load-var http-client-or-sym)]
                  (ctor)))
              http-client-or-sym
              (let [ctor @(dynaload/load-var (configured-client))]
                (ctor)))]
    (when-not (client? c)
      (throw (ex-info "not an http client" {:provided http-client-or-sym
                                            :resolved c})))
    c))

ghadi20:10:46

it is -- we don't expose this functionality yet

borkdude20:10:08

anyway, these are the kinds of spots that need attention when dealing with GraalVM native-image probably. With pprint there was only one or two lines that needed changing and boom, 20mb less binary size

borkdude20:10:36

alter-var-root can help patching these things without even touching the code

ghadi20:10:42

I am more interested in figuring out how to structure truly dynamic code to be graal-sympathetic

ghadi20:10:03

like how could you compile aws-api with the jetty client vs with the something-else-http client

ghadi20:10:02

not dynamic in the produced image, just a higher level front-end API to image generation

ghadi20:10:55

e.g. here is my program, it includes some multimethods, I want you to preload these namespaces which extend the multimethods, this is my entrypoint

ghadi20:10:17

like what is the data that is in @sergey923’s repo?

ghadi20:10:16

{:preload-nses [cognitect.aws.protocols.rest-xml 
               cognitect.aws.protocols.query 
               cognitect.aws.protocols.rest-json]
 :entrypoint latacora.foo/main}

ghadi20:10:08

same stuff with jlink in the jvm

ghadi20:10:39

you tell it the set of modules, but also any dynamic Services that need to be present

borkdude20:10:38

@ghadi I haven't looked into it myself but Quarkus is a JVM framework which has loads of modules/extensions that work with GraalVM. It might have some clues as to how to approach what you're interested in

borkdude20:10:27

@ghadi fwiw, my dynaload variant has a setting for GraalVM: https://github.com/borkdude/dynaload it behaves fully dynamic on the JVM, but less dynamic in CLJS and static in GraalVM native-image (where you're supposed to require the namespaces in a certain order, sure that could be configured using some .edn file)