Fork me on GitHub
#graalvm
<
2021-09-23
>
Karol Wójcik17:09:06

Let me know about libraries you want to use with graalvm https://twitter.com/fierycoddev/status/1441098133555998726?s=21

❤️ 4
Karol Wójcik19:09:35

@U01RXDR9F4M This is the time you can take the list of libraries you're using and post them all 😄

Chris Lowe06:09:47

com.cognitect.aws/*

io.pedestal/pedestal.log            {:mvn/version "0.5.9"}
ch.qos.logback/logback-classic      {:mvn/version "1.2.5"}
com.fasterxml.woodstox/woodstox-core {:mvn/version "6.2.6"}
com.jcraft/jsch                {:mvn/version "0.1.55"}
clj-http/clj-http                   {:mvn/version "3.12.3"}
org.apache.cxf/cxf-core              {:mvn/version "3.4.4"}
io.xapix/paos                        {:mvn/version "0.2.5"}

Chris Lowe06:09:56

Most of my lambdas are on HL 0.2.3. The first 4 entries I’ve configured to run on HL 0.5.0 with the dev graalvm which currently uses the 21.3.x line and forces build-time config.

"--initialize-at-build-time=com.fasterxml.jackson,org.eclipse.jetty,ch.qos.logback,javax.xml.namespace.QName,org.slf4j.LoggerFactory,org.slf4j.impl.StaticLoggerBinder,com.ctc.wstx,javax.xml.stream.FactoryFinder,io.opentracing.util.GlobalTracer,com.sun.org.apache.xerces.internal,javax.xml.parsers.FactoryFinder,com.sun.xml.internal.stream.util.ThreadLocalBufferAllocator,jdk.xml.internal.JdkXmlUtils"
                                                    

Chris Lowe06:09:11

The latter 5 (woodstox and onwards) all gave me pain with getting native build to run correctly and needed careful native/agent code to coax the right build

Chris Lowe06:09:35

to get pedestal logging to build at all on graalvm, I had to disable metrics recording using the follow parameter to graalvm

;; Disable JMX metrics in Pedestal logging (GraalVM build fails, plus JMX metrics are not useful in Lambda)
"-Dio.pedestal.log.defaultMetricsRecorder=nil"

Karol Wójcik06:09:37

Thanks. This is already a lot to be patched 😄

Chris Lowe06:09:43

No worries, and thank you for this!

Chris Lowe06:09:27

an exhaustive list of all my current dependencies is as follows:

camel-snake-kebab/camel-snake-kebab {:mvn/version "0.4.2"}
ch.qos.logback/logback-classic {:mvn/version "1.2.5"}
clj-http/clj-http {:mvn/version "3.12.3", :exclusions [commons-logging/commons-logging]}
com.cognitect.aws/api {:mvn/version "0.8.515"}
com.cognitect.aws/dynamodb {:mvn/version "810.2.801.0"}
com.cognitect.aws/endpoints {:mvn/version "1.1.12.65"}
com.cognitect.aws/s3 {:mvn/version "814.2.978.0"}
com.cognitect.aws/sqs {:mvn/version "811.2.958.0"}
com.cognitect.aws/ssm {:mvn/version "814.2.977.0"}
com.deepbeginnings/eximia {:mvn/version "0.1.3"}
com.fasterxml.woodstox/woodstox-core {:mvn/version "6.2.6"}
com.jcraft/jsch {:mvn/version "0.1.55"}
com.rpl/specter {:mvn/version "1.1.3"}
district0x/graphql-query {:mvn/version "1.0.6"}
io.pedestal/pedestal.log {:mvn/version "0.5.9"}
io.xapix/paos {:mvn/version "0.2.5", :exclusions [commons-logging/commons-logging, log4j/log4j]}
lock-key/lock-key {:mvn/version "1.5.0"}
metosin/jsonista {:mvn/version "0.3.4"}}
net.cgrand/xforms {:mvn/version "0.19.2"}
net.totakke/clj-iso3166 {:mvn/version "0.2.0"}
org.apache.cxf/cxf-core {:mvn/version "3.4.4"}
org.slf4j/jcl-over-slf4j {:mvn/version "1.7.32"} ;; to bridge commons-logging to logback
phrase/phrase {:mvn/version "0.3-alpha4"}
tick/tick {:mvn/version "0.4.32"}
tolitius/xml-in {:mvn/version "0.1.1"}
uk.co.lucasweb/aws-v4-signer-java {:mvn/version "1.3"}

Karol Wójcik06:09:32

Which one is the most important @U01RXDR9F4M

Chris Lowe06:09:00

the first list I posted is in order of importance to me

Chris Lowe06:09:12

clj-easy currently adds cognitect to the Clojure namespaces supplied to the build time parameter, but I had to also add org.eclipse.jetty to the build time list to prevent those scary runtime stack traces we went though last week

borkdude09:09:25

yep, we can fix this by making a clj-easy config for it in https://github.com/clj-easy/graal-config

Karol Wójcik09:09:11

@U01RXDR9F4M whas is necessary for pedestal log to work? Build is successful, but I'm not seeing any logs

Karol Wójcik10:09:38

Oh it probably requires slf4j implementation

Karol Wójcik10:09:50

Forgotten how good logging in java is 😄

Karol Wójcik10:09:15

Pure love for the idea of "let's do a facade for everything guys, ok?" 😂

Karol Wójcik10:09:57

Yeah. Got it working

Karol Wójcik10:09:24

@U01RXDR9F4M how to trigger this metrics recorder?

Chris Lowe10:09:46

not sure how to explicitly trigger the metrics recorder, as far as I’m aware, it’s just “there” and initialises by default

Chris Lowe10:09:17

to that end, it would be triggered by using the logger, I guess

Karol Wójcik10:09:20

What facade do you using for loggin?

Chris Lowe10:09:06

pedestal -> slf4j then using logback for the logger implementation

Chris Lowe10:09:45

pop the above in your resources

Chris Lowe10:09:44

another way to break graalvm build is to specify config scanning in logback.xml. I’ll post an example in a sec

Chris Lowe10:09:24

<configuration scan="true" scanPeriod="5 seconds"> <!-- scan is handy for REPL sessions, but fatal to GraalVM native builds :-) -->

Chris Lowe10:09:48

The following deps and the logback.xml file in resources should be enough to get you logging.

io.pedestal/pedestal.log             {:mvn/version "0.5.9"}
ch.qos.logback/logback-classic       {:mvn/version "1.2.5"}
in your Clojure code:
(:require
    [io.pedestal.log :as log])

(log/error :in 'my-fn :message "this is a message")

Chris Lowe10:09:03

(sorry if that’s teaching granny to suck eggs)

Karol Wójcik11:09:01

Out of curiosity why are you not using slf4j simple?

Karol Wójcik11:09:35

I mean of course i will provide patchem for all of those but still im genuinely curious

mkvlr12:09:03

if these deps all worked (which I haven’t tried, will do so in a few weeks) Clerk should be runnable in native image

org.clojure/java.classpath {:mvn/version "1.0.0"}
org.clojure/tools.analyzer.jvm {:mvn/version "1.1.0"}
org.clojure/tools.deps.alpha {:mvn/version "0.11.905"}
weavejester/dependency {:mvn/version "0.2.1"}
funcool/datoteka {:mvn/version "1.0.0"}
com.nextjournal/beholder {:mvn/version "1.0.0"}
mvxcvi/multihash {:mvn/version "2.0.3"}
lambdaisland/uri {:mvn/version "1.4.70"}
arrowic/arrowic {:mvn/version "0.1.1"}
rewrite-clj/rewrite-clj {:mvn/version "0.6.1"}
funcool/datoteka I plan to replace with babashka.fs

Chris Lowe13:09:18

> Out of curiosity why are you not using slf4j simple? Just familiarity with Logback. I’ll try SimpleLogger as it’s probably more lightweight. For instance, it would do away with a load of XML references.

Karol Wójcik15:09:04

@U01RXDR9F4M simpleogger is easy to bake with Graal. Just include simple logger in initialize-at-build-time

👍 1
Chris Lowe06:09:15

@UJ1339K2B what are the exact build time parameters you’re using for SimpleLogger? It took the following for a simple lambda with only a log statement to work:

"--initialize-at-build-time=com.fasterxml.jackson,javax.xml.parsers.FactoryFinder,com.sun.org.apache.xerces.internal,jdk.xml.internal.JdkXmlUtils"
Strangely, the native compilation didn’t ask specifically for org.slf4j.impl.SimpleLogger, which I was expecting based on your comment. sam local invoke successfully showed log output

Chris Lowe06:09:52

ps - I’m using the dev graalvm image

Karol Wójcik13:09:49

Args=--initialize-at-build-time=org.slf4j.LoggerFactory \
     --initialize-at-build-time=org.slf4j.impl.StaticLoggerBinder \
     --initialize-at-build-time=org.slf4j.impl.SimpleLogger

Karol Wójcik14:09:16

@U5H74UNSF How clerk works? Isn't it dynamically loads arbitrary classes at run time?

Chris Lowe15:09:43

@UJ1339K2B I take it that’s without pedestal logging?

Karol Wójcik15:09:02

But it works with pedestal logging

Karol Wójcik15:09:13

Although don't know how to trigger metrics

Chris Lowe15:09:58

I’ll see if I can send a repro, but it won’t be until tomorrow

Karol Wójcik15:09:38

You mean for metrics? 😄