Fork me on GitHub
#graalvm
<
2020-06-15
>
Kevin12:06:40

Is it possible to create tab-completion with a graalvm compiled project? Like the video below? If so, can someone point me at the right direction?

Kevin12:06:17

Or is that more of an OS think that you need to package?

Kevin12:06:24

ah perfect, no idea why that was so hard for me to find haha

Johannes F. Knauf13:06:38

@kevin.van.rooijen https://github.com/l3nz/cli-matic/issues/39 might be of interest to you as well. If you want to go further into the direction of generating the completion from an abstract option declaration, I can recommend https://github.com/spf13/cobra/blob/master/bash_completions.go as reference.

Kevin13:06:33

Thanks, this is really helpful!

naomarik13:06:56

Added clj-http to my project and getting this error:

Error: No instances of javax.net.ssl.SSLContext are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Tried setting this flag --initialize-at-run-time=.ssl.SSLContext and other things but can't seem to get anything going.

Kevin13:06:50

Assuming you're trying to use the client?

Kevin13:06:20

erm. sorry. this is http-kit, not clj-http

naomarik18:06:14

http-kit works btw on latest dev version

naomarik13:06:30

I can use clj-http-lite but I'm just being stubborn at this point.

borkdude13:06:59

@naomarik isnt't this just these flags?

"-H:EnableURLProtocols=http,https"
       "--enable-all-security-services"

naomarik13:06:32

I'm playing around with attaching a debugger but I have no idea what I'm doing... seeing if I can get a breakpoint somewhere

borkdude13:06:52

if you can figure that one out, it would be great. clj-http-lite has been the escape hatch for it for a while

borkdude13:06:14

which works quite well also btw

naomarik13:06:46

yeah it works for me, but I tend to get stubborn with issues and it's hard to not dig myself in a hole until I get things working the way I want

naomarik14:06:21

Not sure how to do this, in clojure top level require seems to initialize classes which cascades all the way down to the java SSL stuff and trying to wrap http client in a function to dynamically require gets it compiled out of graalvm.

borkdude14:06:39

@naomarik you can't dynamically require something in a graalvm image, at runtime. it has to required at compile time

naomarik14:06:15

yeah, don't have enough knowledge to deal with this

borkdude14:06:36

it boils down to this: eval does not work in graalvm native-image

naomarik15:06:27

I hacked clj-http wrapping the problematic code in a delay and it works, passes all tests too.

naomarik15:06:25

The classes were in a def so always being executed.

borkdude15:06:49

do the tests pass with the native binary?

naomarik15:06:13

I can compile it into my graalvm and it works, is that what you mean?

naomarik15:06:27

lein test passes and my standalone graalvm doesn't complain about anything

borkdude15:06:39

I mean, can you do everything that's done in the units tests, with your graalvm binary? that remains to be seen right?

naomarik15:06:22

not sure how to run all the unit tests in a graalvm binary

borkdude15:06:36

@naomarik one way of testing this is to bind the clj-http functions into babashka and then run the tests using babashka

borkdude15:06:28

I also test other libraries like this such as clj-commons/clj-yaml

naomarik15:06:05

Not sure how to do that ;/ you have an example?

naomarik15:06:17

I can throw my project up on a fork if you want to try it

naomarik15:06:49

Will try that

borkdude15:06:10

@naomarik let me know if you get stuck. this is very interesting

borkdude15:06:39

here are some hints how to get started. most often people don't clone recursively: https://github.com/borkdude/babashka/blob/master/doc/dev.md

naomarik16:06:17

Hey @borkdude, I copied https://github.com/dakrone/clj-http/blob/3.x/test/clj_http/test/core_test.clj into test-resources/lib_tests like like clj-yaml.core-test , I edited babashka.impl.clj-http and main.clj get rid of each error as it came up and got around here:

➜  lib_tests git:(master) ✗ ./clj_http_test
java.lang.Exception: Could not require ring.adapter.jetty. [at /home/naomarik/babashka/test-resources/lib_tests/clj_http/core_test.clj, line 1, column 1]
Error encountered performing task 'run' with profile(s): 'test'
Suppressed exit
➜  lib_tests git:(master) ✗ ./clj_http_test
java.lang.Exception: Unable to resolve classname: java.net.SocketTimeoutException [at /home/naomarik/babashka/test-resources/lib_tests/clj_http/core_test.clj, line 1, column 1]
Error encountered performing task 'run' with profile(s): 'test'
Suppressed exit
before giving up, there's an enormous amount of imports and not sure whether I was putting the right code in the impl file.

borkdude17:06:30

@naomarik I see: the clj-http tests run a server to test clj-http. this requires a bit of a different approach. the server needs to be started in a JVM, and then the clj-http test code needs to run in babashka itself

naomarik17:06:12

yeah seems like a long story 🙂 can't spend time on it anymore unfortunately, but running around babashka source, man you've done a lot of work!

borkdude17:06:28

@naomarik you can try to do a couple of request just outside the tests though: bb -e "(require '[clj-http ...] (client/get ...)", etc

naomarik17:06:20

yup! actually i did that and it works

borkdude17:06:33

@naomarik do you have a link to your fork?

borkdude17:06:46

might be worth doing an issue + PR against the original repo

borkdude17:06:03

@naomarik I also tried it locally myself now with your fork.

$ ./bb "(require '[clj-http.client :as client]) (:status (client/get \"\"))"
java.lang.ClassNotFoundException: org.apache.commons.logging.impl.LogFactoryImpl [at line 1, column 50]
adds about 40MBs to the bb binary (100 mb in total) and eats a lot more ram (if I give it 8GB, it will eat 8GB)

ghadi17:06:00

I would categorically seek to avoid adding clj-http.client to a native image

ghadi17:06:06

it's not that good of an http library

ghadi17:06:13

though many use it

ghadi17:06:35

and it's certainly not designed for native-image (many many deps)

ghadi17:06:27

40MB is a lot

borkdude17:06:07

saw a similar thing with hato. clj-http-lite might be a nice middle ground for native-image CLIs

borkdude17:06:28

or if shelling out to curl is fine, then https://github.com/borkdude/babashka.curl is probably the lightest solution

borkdude17:06:18

it's still cool that naomarik got it to compile though, kudos

naomarik18:06:23

oh strange, for my project clj-http compiles to 66MB, clj-http-lite is 57MB

naomarik18:06:08

Without either it's 56MB, so clj-http adds 10MB for me.

naomarik18:06:26

Using GRAALVM_VERSION="20.1.0" , openjdk version "11.0.7" 2020-04-14 under WSL2 ubuntu

borkdude18:06:39

that's weird. I'm on mac here

borkdude18:06:23

These are binaries built on CI:

[8:02 PM] [linux-static - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19]: 
[8:02 PM] [linux-static - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19] binary size: 105772312
[8:03 PM] [linux - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19]: 
[8:03 PM] [linux - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19] binary size: 110654720
[8:05 PM] [macos - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19]: 
[8:05 PM] [macos - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19] binary size: 103977664

borkdude18:06:27

all over 100MB

borkdude18:06:51

without clj-http ~60MB

borkdude18:06:07

maybe you did some other things apart from the patch @naomarik?

naomarik18:06:22

no nothing, are you using the latest graalvm?

naomarik18:06:53

i believe the graalvm I'm using is java 11 as well

naomarik18:06:10

not sure if that makes a difference

borkdude19:06:41

I'm using the same version you are using. Maybe you already had a lot of overlapping deps with clj-http and my project didn't