Fork me on GitHub
#graalvm
<
2020-04-14
>
David Pham11:04:09

If I understand correctly, we can compile Clojure code into LLVM bytecode with GraalVM right?

David Pham11:04:26

If this is the case, we could also take this code and convert into Webassmbly?

jaihindhreddy11:04:42

I dunno whether that's possible and if so, how straightforward it is to do. But, I'm pretty sure it won't be practical in terms of bundle size (assuming browser use) because we'd have to ship the runtime (stuff like GC). May I know the use case you have in mind?

David Pham11:04:44

I just want to write a simple numerical function to optimize some computation on the front end

David Pham11:04:03

It would be a rolling average function like m_n = m_{n-1} + x_n/n - x_{n-w-1}/n

David Pham11:04:03

I could try and write it into C++, but it was more an idea.

👍 4
jaihindhreddy11:04:50

I heard Rust has a good compile-to-wasm story. That might yield a much smaller output.

David Pham11:04:35

I will try. I never coded in rust though haha

sogaiu12:04:16

if you go for rust, i highly recommend checking out this chapter before diving in: https://www.oreilly.com/library/view/programming-rust/9781491927274/ch04.html

David Pham13:04:04

I just want to write (reductions + coll) in rust, I hope it won’t translate into 100s of lines!

jaihindhreddy15:04:37

Probably not. Rust is pretty concise. BTW we can take this to #other-languages or #off-topic because it isn't relevant to #graalvm or Clojure.

lukasz16:04:15

Is it safe to assume that if my Clojure app uses only the JDK (and Clojure's std lib) it should be straight forward to compile it using native-image?

ghadi16:04:00

yes, but it depends™

ghadi16:04:38

I'm not sure of all the edge cases, but it's way more likely that you'll be good than if you incurred a bunch of random dependencies

lukasz16:04:32

I thought that will be the case - all I have to do is to periodically fetch data from an HTTP endpoint, parse the json and send some of the data over to a statsd server - seems like all of these should be doable with just the JDK - JSON is probably the harder part, but I've noticed that babashka has cheshire as a dependency so It Should Work:tm:

ghadi16:04:34

clojure.data.json should be fine

ghadi16:04:47

the http part is trickier

lukasz16:04:02

plain slurp seems to be working based on my testing via the repl

borkdude16:04:09

@lukaszkorecki writing this as a simple babashka script should also work. equally fast startup, no compilation needed

lukasz16:04:18

@borkdude can babashka do loops, as in (while true (do (something) (Thread/sleep 10)) - I haven't had the time to play with it yet

lukasz16:04:32

This Changes Everything

lukasz16:04:40

the udp/statsd part will be fun

borkdude16:04:13

@lukaszkorecki I've included babashka.curl inside bb. curl can do amazing stuff, talk http over unix sockets for example. don't know about the udp stuff you're talking about

lukasz16:04:52

Oh, it would be a simple statsd client - something I wrote before many times (in Common Lisp, Chicken Scheme, C etc etc )

lukasz16:04:01

Nothing complicated

borkdude16:04:10

is that doable with a socket? babashka also includes http://java.net.Socket etc

lukasz16:04:21

Yeah, http://java.net.Socket should be enough

lukasz16:04:54

statsd itself has a clojure implementation based on it https://github.com/statsd/statsd/blob/master/examples/statsd.clj

borkdude16:04:02

@lukaszkorecki cool. and if you decide you want to run as a native image anyway, someone made a migration script from bb to native-image: https://github.com/MnRA/nativity 🙂

lukasz16:04:27

yeah, I saw that yesterday - looks really neat

lukasz16:04:38

I'm really trying to avoid writing Go right now ;-)

borkdude16:04:43

hmm, I don't think DatagramSocket etc is in bb right now: https://github.com/borkdude/babashka/blob/36c163a4449e43cc99be2ce5b32794d854d4ab68/src/babashka/impl/classes.clj#L56 but it's just a matter of adding it to that list.

borkdude16:04:02

I'll take a look tomorrow if I can get that library working with bb

lukasz16:04:24

Oh thanks - I'm drastically short on time, otherwise I'd look into it

borkdude16:04:02

@lukaszkorecki was fun listening to the podcast (the REPL) btw!

lukasz16:04:22

thanks! I tried my best to stay on topic(s)

borkdude17:04:45

@lukaszkorecki Are you on Mac or Linux?

borkdude17:04:27

@lukaszkorecki With a few of those Datagram classes added, this now works:

$ ./bb -e '(load-file "/tmp/statsd.clj")' -e "(require '[statsd-client :as c]) (c/increment :foo)"
(where statds.clj is the example you linked to earlier)

borkdude17:04:44

This is a linux binary: https://6696-201467090-gh.circle-artifacts.com/0/release/babashka-0.0.85-SNAPSHOT-linux-amd64.zip And this is a macos binary: https://6697-201467090-gh.circle-artifacts.com/0/release/babashka-0.0.85-SNAPSHOT-macos-amd64.zip with these classes added. Let me know if this is useful to you. If yes, I'll keep 'em in for the next release 😃.

lukasz17:04:49

Linux works, nobody's going to run this on a mac

lukasz17:04:09

thanks a lot @borkdude - I'll see if I can stitch it all together today!

borkdude17:04:45

@lukaszkorecki If this is going to run on Linux alpine, let me know, then you'll need the static binary

lukasz18:04:54

target container size is not that important, it's more about the memory/cpu footprint of the tool itself

lukasz18:04:03

so I'll most likely end up with a Debian image

lukasz18:04:03

if it all works - I'm going to write about how we are monitoring our services in ECS with graphite and statsd

borkdude19:04:34

@lukaszkorecki Did an additional test with https://www.npmjs.com/package/statsd-logger

$ statsd-logger
Server listening on 0.0.0.0:8125
StatsD Metric:  foo:1|c

lukasz19:04:58

Really cool - I already got a POC - just looking at CPU/RAM usage

borkdude19:04:13

$ ./bb -e '(load-file "/tmp/statsd.clj")' -e "(require '[statsd-client :as c]) (while true (c/increment :foo) (Thread/sleep 100))"
$ statsd-logger
Server listening on 0.0.0.0:8125
StatsD Metric:  foo:1|c
StatsD Metric:  foo:1|c
StatsD Metric:  foo:1|c
...

borkdude19:04:54

@lukaszkorecki If I put (System/gc) at the end of the iteration, it runs pretty consistently with 11 MB of memory (reported in Activity Monitor on a Macbook Pro)

lukasz19:04:37

Nice! Let me wire polling the HTTP server and reporting metrics via UDP to see how it behaves

lukasz19:04:50

thank you for the quick turnaround @borkdude

lukasz22:04:28

It works 🎉 but leaks memory somewhere, I need to dig into it - thank you for your help @borkdude

lukasz22:04:29

The fix was (obviously) to move the loop out of bb and leave it to the shell 🤷