Fork me on GitHub
#babashka
<
2022-11-20
>
staypufd00:11:23

@borkdude You may like this talk, in case you are thinking of Babashka on WebAssembly at some point. Given by guy who worked on V8 at Google. https://youtu.be/P_7ptIBg4PY

borkdude10:11:13

Thanks, added to my watch list

skylize03:11:27

Can Babashka run Kaocha? If so, how do I start it with a Task? I have a deps.edn that includes

;; deps.edn
{...
 :aliases {...
           :test {:extra-paths ["test"]
                  :extra-deps {lambdaisland/kaocha
                               {:mvn/version "1.71.1119"}
                               ...}}
           :runner {:main-opts ["-m" "kaocha.runner"]}}}
And a bb task that runs a test suite through Clojure,
;; bb.edn
{...
test-clj (clojure "-M:test:runner" "clj")}
which I run with bb test-clj. How do I run the same test suite, but in Babashka instead of Clojure? (The point being to verify Babashka compatibility.)

borkdude10:11:00

You can try it like this: bb.edn

{:deps {lambdaisland/kaocha {:mvn/version "1.71.1119"}
        org.babashka/spec.alpha {:git/url ""
                                 :git/sha "1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78"}}
 }
bb -m kaocha.runner
I came across one missing class: java.lang.Error - I'll add that to bb. Continuing with bb master:
clojure -M:babashka/dev -m kaocha.runner
Type:     java.lang.Exception
Message:  Unable to resolve classname: clojure.lang.DynamicClassLoader
----- Stack trace --------------------------------------------------------------
dynapath.defaults            - dynapath/defaults.clj:4:3
dynapath.util                - dynapath/util.clj:3:3
kaocha.classpath             - kaocha/classpath.clj:5:3
kaocha.testable              - kaocha/testable.clj:3:3
kaocha.plugin.capture-output - kaocha/plugin/capture_output.clj:2:3
kaocha.report                - kaocha/report.clj:92:3
kaocha.config                - kaocha/config.clj:2:3
kaocha.api                   - kaocha/api.clj:4:3
kaocha.runner                - kaocha/runner.clj:5:3
user                         - <expr>:1:10
It seems kaocha is doing stuff with classloaders. Not sure why, but in bb this doesn't work

borkdude10:11:03

kaocha is quite a complex project. If you can't get it to work, I recommend running the cognitect test runner which works with bb today

skylize12:11:18

Thx. So master has java.lang.Error now? I'll give it a try. Pretty sure I read something in the Koacha docs about disabling the classloader stuff.

skylize14:11:00

Ran the script to install bb master, and still see java.lang.Error missing. I found the info about Koacha's ClassLoader. It is there to allow the option of specifying test source paths in test.edn instead of deps.edn. It is easily disabled with a config option on the test suite.

;; test.edn
#kaocha/v1
 {:tests [{:id :unit
           :skip-add-classpath? false}]}

borkdude14:11:05

What did you do to install bb master ?

skylize14:11:37

bash < <(curl -s )

borkdude14:11:10

You need to specify the snapshot version

skylize14:11:50

where do i find the snapshot version?

skylize16:11:31

Got it. Thanks.

skylize16:11:27

😞 First try, it is still crashing on code about the ClassLoader. Hopefully I am just misunderstanding where/how to specify the config option. Otherwise, I guess I'll file an issue with Kaocha, and hope. Then use cognitect in the meantime.

lispyclouds16:11:43

You could try asking in the #CCY2V0U6A channel too?

borkdude16:11:25

sorry, was afk but yes that's where you should use the version from

skylize04:11:40

Custom build of bb with DynamicClassLoader class, just to see how far I can get when importing doesn't break, and .. Crashed again, on code still trying to do ClassLoader shenanigans, despite my config saying "don't do that". 😒

borkdude10:11:14

@U90R0EPHA To make progress on this, it's probably best to run with a local checkout of kaocha and remove stuff that works around the classloader stuff

sheluchin14:11:32

I'm getting many files like .babashka-pod-2027225.port in my project root where my bb.edn lives. Not sure where it comes from. Is there some explanation of these somewhere? Or a guess of why they might be hanging around?

Apple14:11:06

it's probably just a number inside indicating the repl port. should be safe to delete.

sheluchin14:11:23

Yes, it is that, but is it normal that they're hanging around like this, or does it indicate some problem in my setup?

dpsutton14:11:00

The polite thing to do is mark those files as deleteable when the vm exits. If you have any processes running, stop them and see if the files remain. If so you should raise an issue I think

borkdude16:11:52

@UPWHQK562 which pod causes this?

sheluchin16:11:34

I think it's {epiccastle/spire {:version "0.1.1"}}, but I know I should be moving to https://github.com/epiccastle/bbssh, so maybe it's a moot point... just haven't had the time to upgrade yet.

borkdude16:11:12

Perhaps make an issue there : the way it should be implemented is: when their process shuts down, they should remove the file. Indeed what @U11BV7MTK says: mark it as delete on shutdown

sheluchin16:11:03

Okay, so it's definitely an irregularity then. Alright, I'll try to confirm/get more detail about it and will report. Thanks, all.

Crispin13:11:01

Right gotcha.

Crispin13:11:12

There is a shutdown hook to delete it. I wonder why its not working? https://github.com/epiccastle/spire/blob/master/src/clj/spire/pod/core.clj#L105-L108 :thinking_face:

Crispin13:11:49

I will do some testing thanks @UPWHQK562

borkdude13:11:38

if you implement the "shutdown" operation, then the pod is shutdown more gracefully, might also help

Crispin13:11:33

This is the pod mainline, not running in babashka. I thing shutdown operation will be the best thing.

borkdude13:11:37

not sure what you meant with "this is the pod mainline ..."

Crispin13:11:52

I don't have babashka.fs as a dep atm in the code.

Crispin13:11:39

its the clojure graal compiled code

borkdude13:11:46

what I meant was: you can just do what that function does (or add fs as a dep)

Crispin13:11:10

yeah I figured looking into what its doing under the hood

Crispin13:11:35

ok that .deleteOnExit works.

sheluchin19:11:08

Thanks for checking this out, @U1QQJJK89. Not sure if you've concluded this, but just FYI, I updated the repro to use (pods/load-pod 'epiccastle/spire "0.1.2") and I'm still getting the port files left over in my directory after the process is done.

Crispin13:11:47

@UPWHQK562 ah damn. Was working when running the pod under JVM. But as graal native-image binary deleteOnExit doesn't work. I've come up with something now that seems to work with native image to. I will re-tag and release 0.1.2. Hold on.

borkdude13:11:33

Should also work in a graal binary, but did you also implement the shutdown op?

Crispin13:11:54

I tried the shutdown too. It was never called.

borkdude13:11:12

it could be that in a shutdown that is forced, the delete on exit doesn't have time to execute

Crispin13:11:15

It's strange. I don't even reach the EOF exception

borkdude13:11:24

if you implement the shutdown op, it should imo work

Crispin13:11:35

yeah it didnt work

Crispin13:11:52

but what Im doing now is deleting it after socket is established, while its still running

borkdude13:11:28

ok, that should work too... still weird that the shutdown op didn't work, it should.

Crispin13:11:30

after the first read

Crispin13:11:21

yeah... something strange going on. And it worked on JVM which is doubley strange. Don't have time to dig in atm. This early delete seems to work for now.

Crispin13:11:34

ah do I have to set an option? to get the shutdown hook

borkdude13:11:02

yes, you need to declare it in your describe message that you implement the shutdown op. RTFM dude ;)

borkdude13:11:07

hehe kidding

Crispin13:11:15

haha no time! 😉

Crispin13:11:41

skim reading is the new reading

Crispin13:11:01

let me give it a whirl

Crispin13:11:49

"If the pod does not support the shutdown op, the pod process is killed by the pod client"

Crispin13:11:12

that explains it. It's nuking it before it can do anything. Wow, reading is quite useful, hey 😄

borkdude13:11:48

> The optional ops value communicates which ops the pod supports, beyond describe and invoke. It is a map of op names to option maps. In the above example the pod declares that it supports the shutdown op.

borkdude13:11:15

the docs could be clearer of course, sorry for that ;)

borkdude13:11:07

probably the pods thing can also remove the file after it connects, this seems more reliable than asking people to do this

borkdude13:11:12

issue welcome as well

borkdude13:11:58

I think bb could mark the file as delete on exit

borkdude13:11:19

and when it doesn't exist anymore, that won't crash either I think so it will be a double safety measure

Crispin13:11:02

ok, compiling with a shutdown hook now...

Crispin13:11:46

yeah. I will test .delete ing a non existent file and if there is no exception then I could patch bb to delete it after connection

borkdude13:11:35

I think just marking it as delete on exit in bb will be sufficient, not actually deleting it

Crispin13:11:21

yeah. its probably good to not delete it early in bb, as it is the pod that makes it

Crispin13:11:39

who knows what the pod is doing

Crispin13:11:43

but makes sense in cleanup

Crispin13:11:01

alright, got the shutdown message calling me. and file deleted nicely from that message. 🎉

👍 1
Crispin14:11:14

@UPWHQK562 I have rebuilt and rereleased 0.1.2 with a working file cleanup. You will need to delete your local pod repository ~/.babashka/pods/repository/epiccastle/spire/0.1.2/ to have babashka fetch the new 0.1.2 binary.

sheluchin22:11:26

Thanks, @U1QQJJK89. Looks like it worked! I closed the issue.

👍 1