Fork me on GitHub
#babashka
<
2020-04-08
>
sogaiu00:04:17

@borkdude @bherrmann @dominicm success with building and running babashka on aarch64

rovanion06:04:08

I can wholeheartedly recommend the Azul Zulu Embedded JVM for (low performance) ARM systems. It is fast enough that developing normal Clojure on a Raspberry Pi Zero was actually feasible.

sogaiu07:04:41

ah, that sounds nice, thanks for the recommendation @rovanion

rovanion07:04:18

Does anyone have a pointer for reading the output of a ProcessBuilder thus far without blocking?

borkdude07:04:11

@rovanion the Process returned by ProcessBuilder has an outputstream you can read from

rovanion07:04:16

Yes, but how do I read the output written by the process so far, and not get blocked waiting for more?

rovanion07:04:30

(On my system it's the InputStream that carries the stdout of the process, but I assumed that was what you meant or that I'm confused, either way it was not important to the question at hand.)

borkdude07:04:39

@rovanion you can maybe use (pos? (.available input-stream)) to poll for more input.

borkdude07:04:00

yeah, the naming of those streams is confusing

rovanion07:04:22

Right, will see if I can build something around that.

borkdude07:04:01

@rovanion you can also read input in a core async thread and then put the input on a channel, or read input in a future and handle it there

dabrazhe09:04:15

Hi. I am wondering if Iโ€™d be feasible to release Babashka as AWS lambda custom runtime ? For simple lambdas that could benefit from the fast start up

borkdude09:04:43

@dennisa see https://github.com/dainiusjocas/babashka-lambda (made by @dainius.jocas) for an example

dabrazhe09:04:07

Aha. Does it package Graal Vm?

borkdude09:04:31

@dennisa No. If you only use babashka scripts, you don't need GraalVM, only the bb binary, which you can download from Github releases

borkdude09:04:48

or if you need Docker images, you can use the babashka Docker base image

borkdude09:04:26

Note that babashka v0.0.80 is the newest and also has babashka.curl which uses curl to do HTTP requests. You might also be able to leverage that instead of clj-http-lite which was used by Dainius. Although he mentioned that shelling out to curl did have some slowness associated with it, which could be related to environment specific settings like memory availability

dabrazhe15:04:57

Cool, this is great, thanks!

rovanion10:04:38

I've come to a point where I can can read only the available bytes. But I'm allocating a byte array for each read I do which is inefficient and meh: http://paste.ubuntu.com/p/yBPHNYCt75/

rovanion10:04:24

What I really want is a BufferedReader which doesn't block when there is no more to read from the stream...

rovanion10:04:26

Well, when there is more to read from the stream but not right now.

borkdude10:04:37

@rovanion I'm not sure how your paste relates to what you are saying here

rovanion10:04:22

Ah, wrong file...

borkdude10:04:43

I think this is incorrect: stdout (.getErrorStream process) stdin (.getErrorStream process) stderr (.getErrorStream process)

borkdude10:04:44

You can do (pos? (.available is)) to detect is something is available and then either read or not

borkdude10:04:04

@lilactown I've added something like this now in the sigint-handler branch:

(require '[babashka.signal :as signal])

(def proc (let [pb (ProcessBuilder. ["yes"])
                p (.start pb)]
            p))

(signal/add-sigint-handler! :quit (fn [] (println "bye1")))
(signal/add-sigint-handler! :quit2 (fn [] (println "bye2")))
(signal/add-sigint-handler! :quit3 (fn [] (.destroy proc)))

(.waitFor proc)
Welcome to give some feedback on it. Note that on my machine, even without the sigint-handlers, the proc goes away, i.e. I see no yes process running after the script is killed. Binaries are available in #babashka_circleci_builds. Maybe the signal handler functions should at least receive the key with which they were registered. Maybe that's not necessary because you can just close over your own key if you want.

borkdude13:04:34

@lilactown I also tested the above on linux: the process goes away when I kill the script (without adding a sigint handler)

rovanion15:04:20

A working implementation of the read as much data as is available code I've been on about the whole day.

rovanion15:04:53

Associated tests.

borkdude15:04:07

@rovanion Cool! Note that .waitFor also returns the exit code so you could also do {:exit-code @(:future-exit-code process)} possibly

rovanion15:04:00

Yup, those should be equivalent.

borkdude15:04:51

@lilactown This script is now in the tests:

(require '[babashka.signal :as signal])

(signal/add-interrupt-handler! :quit (fn [k] (println "bye1" k)))
(signal/add-interrupt-handler! :quit2 (fn [k] (println "bye2" k)))

(System/exit 0)
It prints:
bye1 :quit
bye2 :quit2
API is still up for discussion. I think a better name might actually be add-shutdown-hook or something

Mitch15:04:01

I'm not able to (require '[clojure.data.csv]) when using bb --nrepl-server . Is there a special trick I am missing?

borkdude15:04:33

@mitchell_clojure works over here:

borkdude15:04:43

are you connected to the right server?

Mitch15:04:56

thanks, I have a hunch that it might have something to do with my local maven repo

Mitch15:04:06

it is unable to find clojure.data.csv on my classpath

borkdude15:04:56

@mitchell_clojure clojure.data.csv is built into babashka, so you don't need a classpath

Mitch15:04:58

I'm just running bb --nrepl-server. Do I need to do anything else?

borkdude16:04:38

@mitchell_clojure cider-connect

Mitch16:04:52

oh yup, have done that too

Mitch16:04:47

I'm on version 0.0.80, not sure if anything has changed since that might affect this

borkdude16:04:10

cider connect to what port?

Mitch16:04:17

localhost:1667

borkdude16:04:35

v0.0.80 is what I'm running here as well. if the buffer in which you are evaluating associated with that cider session?

Mitch16:04:35

according to the output from bb

Mitch16:04:51

yes it is

borkdude16:04:04

are you running any other nREPL servers?

Mitch16:04:30

yeah, a few

borkdude16:04:50

just to make sure, restart emacs and kill all other sessions

Mitch16:04:08

Okay will do, have a meeting right now but will let you know the resolution

Mitch16:04:09

thanks again

borkdude17:04:36

@lilactown Ah, I found something much simpler without introducing a new namespace:

(import '[java.lang Runtime])

(-> (Runtime/getRuntime) (.addShutdownHook (Thread. #(println "bye"))))

@(promise)
Press ctrl-C and it prints bye, both in Clojure and bb ๐Ÿ™‚

lilactown18:04:16

Ah that's great

borkdude18:04:10

@lilactown Btw, could you verify the behavior of this one?

(def proc (let [pb (ProcessBuilder. ["yes"])
                p (.start pb)]
            p))

(.waitFor proc)
When I press ctrl-c, the yes process isn't running anymore afterwards

lilactown18:04:49

I'm not at my computer today but I'm setting a reminder to check tomorrow

lilactown18:04:24

I'm specifically using socat in my script which maybe doesn't behave well

lilactown17:04:45

@borkdude what Iโ€™m finding is that running socat and then calling .waitFor on the process obj does kill socat when I CTRL-C

lilactown17:04:26

however, not .waitFor ing causes the bb process to exit but socat to stick around

lilactown17:04:55

doing the same with yes does not exhibit this behavior; the yes process does not persist whether or not I .waitFor

borkdude17:04:33

hmm, ok. a similar issue like @U0510902N reported apparently. I wonder if adding the shutdown hook helps here as well.

lilactown17:04:14

yeah itโ€™s strange. creating multiple socat processes and .waitFor ing one of them, causes all of them to exit when I CTRL-C the bb process. however, not .waitForing causes all of the processes to persist after the bb script is done evaluating.

lilactown17:04:18

so in the case Iโ€™m running into, it seems like CTRL-C propagates the terminate signal to subprocesses, but the script exiting does not? Iโ€™m a little out of my depth now so not sure if my verbiage is right

Mitch18:04:26

following up on my issue earlier, I restarted everything and the nrepl server appears to work well. I just get a few warnings about being unable to determine nrepl/clojure versions. Issue was with helm

Mitch18:04:12

One small thing I noticed was that (do (println "test") (System/exit 0)) prints then exits when using the socket repl, but does not appear to print when using nrepl

borkdude18:04:03

how it's currently implemented: the entire expression is evaluated within a with-out-str and then that output gets sent to the client. but as the server is already killed by then, you won't ever see the println

Mitch18:04:13

Oh okay, so I do not have to worry that previous expressions weren't evald

Mitch18:04:47

Awesome, thanks again. I showed bb at a lunch and learn today and there is definitely some buzz at our place, especially with nrepl up and running

borkdude21:04:02

^ @lilactown