Fork me on GitHub
#babashka
<
2019-12-21
>
borkdude15:12:02

@sogaiu I've made some progress on JDK 11, but I'm stuck on java.nio: https://github.com/borkdude/babashka/issues/120

borkdude16:12:04

Note that I can't reproduce it in Java yet and this would be good for an issue

sogaiu18:12:42

i will take a look -- thanks for filing.

borkdude18:12:05

it would be interesting to see if this is reproducable on linux as well (using the issue I posted)

sogaiu18:12:39

ah, i can probably check that

sogaiu18:12:23

compilation was successful for me. i checked my java version before and after compiling and made sure to use a java 11 graal, fwiw.

sogaiu18:12:33

the resulting bb does start up too

sogaiu18:12:50

the output you posted specifically mentions llvm iiuc: https://pastebin.com/XpBSi1bW

sogaiu18:12:50

anything i should double-check to make sure of my verification procedure?

borkdude19:12:03

You used the same jar as in the issue?

borkdude19:12:03

ah, now that's interesting

borkdude19:12:43

Not sure if it's relevant

borkdude19:12:21

It seems they are fixing a bunch of issues in this area

borkdude19:12:31

So I'll wait for their feedback or 19.3.1 and then try again

borkdude19:12:27

The clojure.lang.Reflector patch seems to have worked

sogaiu19:12:34

i guess we are less than a month away from 19.3.1 -- jan 14th (https://www.graalvm.org/docs/release-notes/version-roadmap/)

sogaiu19:12:41

if i had a better setup, i might consider building off of their current work -- that would let us check whether some of their fixes actually work. i have this feeling that building is likely to be resource and time intensive though...

borkdude19:12:29

hmm, maybe if we ask in their chat channel, they can provide master builds?

sogaiu19:12:03

i think that would be helpful for people trying to test -- don't know how much work that would be for them though...

borkdude19:12:34

if they have a github action, we could maybe download the artifacts from that

sogaiu19:12:09

would be nice if they had something like nightly snapshots

borkdude19:12:52

I asked them now

sogaiu19:12:55

thanks! i hope for everyone's sake that it comes to be before too long.

borkdude19:12:58

I was asking about this in the #general channel of their slack, but like you say, it's probably a time consuming rabbit hole

borkdude19:12:51

The only reason I'm holding off on adding clj-http.lite is that JDK 11 might add something interesting, it has a built-in HTTP client with async support. But a working JDK 11 might take some time. Do you think it's worth the wait, or shall I go ahead with clj-http.lite?

borkdude19:12:43

It's not that curl offers an async API right?

nate20:12:29

Is it bad to have both?

nate20:12:16

I think having a solution now is useful and adding another later is more choice.

borkdude20:12:48

I guess not, but I don't want to clutter babashka and make too many breaking changes I guess?

borkdude20:12:38

Adding support for the raw classes is always an option. I was looking at this lib which leverages the new JDK 11 async stuff: https://github.com/schmee/java-http-clj

borkdude20:12:02

I personally don't expect to need many async requests in scripting, but who knows

borkdude20:12:03

There is also core async which can be used for asychrony

sogaiu20:12:15

indeed -- was going to suggest that, but wasn't quite sure what you meant by async.

sogaiu20:12:52

if it's not too hard, i wonder if adding some interface whose implementation might be swapped out later might be a possibility.

borkdude20:12:30

well, I think an async interface is going to be different anyway

nate20:12:56

I think, if anything, http request handling would be a candidate for multiple libs. Simple with http-lite, a sync with the jdk 11 stuff.

nate20:12:37

But these are just my thoughts. No pressure. Glad to have what we have now.

sogaiu20:12:07

on a related note, in general when faced with "should x be added", one of the concerns seems to be "is the specific implementation what we're going to want eventually?". as a hedge against later changes and related incompatibilities, some amount of shielding using an interface might be helpful?

sogaiu20:12:50

ofc, some thought about the interface becomes necessary...but may be that's not so bad?

borkdude20:12:05

@sogaiu Some amount of shielding takes away the option to run the scripts on the JVM

👍 4
nate20:12:24

The problem with adding another interface is that makes it babashka only. And compatibility with clojure is a great benefit.

borkdude20:12:36

and I want to give people something they're already used to. most people already know clj-http

sogaiu20:12:13

i guess it would be nice if some of the potential additions could not necessarily be built in.

sogaiu20:12:17

but still usable

borkdude20:12:02

you mean like a library

sogaiu20:12:12

i agree that clojure compatibility is a good thing (tm)

sogaiu20:12:00

keeping the core smaller if possible seems preferable

borkdude20:12:14

maybe there can be support for the classes that clj-http-lite uses, so you can make your own implementations that look like it, but it's not so trivial to make it idiomatic for clojure like that library does I think

borkdude20:12:39

but worth a try

sogaiu20:12:05

may be there are easier things to test the idea out on 🙂

sogaiu20:12:56

so that's 3 things? * http://java.net.URI * http://java.net.URL * http://java.net.HttpURLConnection looks like good generic stuff to have (at least the first 2)

sogaiu20:12:22

don't know enough about the 3rd one

borkdude21:12:46

I'll give that a try

borkdude21:12:19

This seems to work so far, except that defn- doesn't exist yet:

(load-file "/Users/borkdude/Downloads/clj_http_lite_core.clj")
#'clj-http.lite.core/request

borkdude21:12:48

And clj-kondo suggested that two of the three classes aren't even used 😉

borkdude22:12:57

so I guess the concept works 🙂

sogaiu22:12:25

sounds promising -- will try

borkdude22:12:52

I found several issues in sci that I will fix later, e.g. the absence of defn- and some ns form parsing error

borkdude22:12:05

but I adapted the library to work around that for now, minor issues

sogaiu22:12:06

cool -- nice bonus to have reasons to add defn- and discover a parsing error.

sogaiu22:12:00

so apart from git pull, switching to the jdk-url branch and git submodule update, is there some other prep i need to do?

sogaiu22:12:27

because i get:

Syntax error (ClassNotFoundException) compiling at (babashka/impl/classes.clj:153:16).
javax.xml.bind.DatatypeConverter

sogaiu22:12:31

may be i missed something?

sogaiu22:12:47

may be i need to clean

borkdude22:12:59

maybe your java doesn't have that class?

borkdude23:12:09

what jvm version are you using?

sogaiu23:12:11

ah, it's probably that i'm using 11

sogaiu23:12:41

hurray! that was it.

sogaiu23:12:42

nice work!

borkdude23:12:22

since babashka already includes the Base64 class

sogaiu23:12:28

it seems java.util.Base64 was added in jdk 8(?) and is the way of the future regarding base64 in java iiuc

sogaiu23:12:20

it's nice that there weren't many changes necessary

sogaiu23:12:47

may be i can grumble a bit about how the reader conditionals for clojure don't feel like an "open system" 🙂

sogaiu23:12:51

do you know if things other than :clj, :cljr, :cljs, and :default are just ignored by the various implementations?

borkdude23:12:38

everything is ignored other than what the current runtime is interested in

sogaiu23:12:24

may be that's actually a pretty open approach afterall.

borkdude23:12:59

Btw, adding more and more classes to the reflection config does make the binary a little bigger every time. It has grown about 5MB since the last few releases. I'm not sure if I should worry about that. Zipped it's still around 10MB.

sogaiu23:12:31

i think it's still ok 🙂

borkdude19:12:28

Awesome article mentioning several Clojure "startup" solutions: https://stuartsierra.com/2019/12/21/clojure-start-time-in-2019

Lyn Headley20:12:25

any plans to enable babashka scripts to be distributed as single binaries?

borkdude20:12:29

That had not crossed my mind yet.

borkdude20:12:36

Why would that be useful to you? And do you have any ideas on how to implement this?

Lyn Headley20:12:59

I am writing a script and getting delusions of grandeur that others might want to easily install and use it.

Lyn Headley20:12:35

no idea how to implement except maybe automating a process which writes scripts into static strings that get compiled into graalvm-produced binary.

borkdude21:12:15

I guess you could also package it inside a docker container 🙂

borkdude21:12:35

(which seems a bit heavy for that purpose maybe)

sogaiu21:12:50

i like the basic idea -- no good ideas about how to go about it yet though

borkdude21:12:06

Just scp-ing a zip file with all the stuff in it, unzipping it and then run it might not be too bad. You could probably even write a script for it 😉

sogaiu21:12:15

i am a sucker for the single-executable idea 🙂

sogaiu21:12:34

i wonder whether there's some stuff here: https://github.com/BrunoBonacci/lein-binplus that might be usefully applied to this situation

sogaiu21:12:37

i guess that approach assumes java exists on the system and is on one's path

Lyn Headley21:12:50

I just threw a pmap into my script and the running time went from 1m44s to 33s!

borkdude21:12:08

What are you doing in parallel?

Lyn Headley21:12:19

calling the github api

nate21:12:33

Wow. Awesome!

Lyn Headley22:12:49

:status 403 :message You have triggered an abuse detection mechanism. Please wait a few minutes before you try again. 😁

sogaiu21:12:31

that's neat!

deleted23:12:58

@borkdude confirmed that *file* works! 😄

borkdude23:12:16

Cool, thanks! I saw your PR. I wasn't sure if you used the ./update-babashka script, so I just did it myself.

borkdude23:12:11

Next time if I forget to update the brew package, just throw a message in the channel

borkdude23:12:44

It's easier for me to run the script than to accept a PR, because I have to check it anyway 🙂