Fork me on GitHub
#babashka
<
2022-10-25
>
steveb8n08:10:56

Q: where is a good example of github actions using bb? I’m about to add CI to a new OSS project

Ferdinand Beyer09:10:04

Have a look how Portal uses bb tasks in GitHub actions — I got some inspiration from that! https://github.com/djblue/portal I like that everything is bundled in BB tasks, and the Github workflow runs those.

🙏 1
borkdude08:10:59

@steveb8n setup-clojure is the all-in-one which can also install bb (and clj-kondo): https://github.com/DeLaGuardo/setup-clojure/ A dedicated one: https://github.com/turtlequeue/setup-babashka

steveb8n08:10:06

perfect. thanks. I’ll share a blog post soon about the new repo

🎉 2
borkdude14:10:59

Native image 22.3 is out with support for Java 19. This means we can have virtual threads in #babashka - if we move to Java 19 :) https://www.graalvm.org/release-notes/22_3/#native-image There is one caveat: > Note that the java.io.Console API does not work properly on JDK 19: The information whether or not a Console is available is wrongly computed at build time. So now the question is: do we wait for Java next LTS, or do we go with Java 19 for the next bb. Another option is to update the jdk19-loom branch so you can compile your own bb with virtual thread support. Follow up in 🧵 .

👀 1
borkdude14:10:15

I would say, let's stay on the stable and let early adopters experiment with the jdk19-loom branch

👆 3
lread15:10:51

I guess also: preview features are "subject to change"

mkvlr15:10:59

I’m for defaulting to loom. How would this console bug affect me?

borkdude15:10:28

The console bug would affect you with tools that e.g. look if you are connected to a console and then print colorized output instead of raw output

mkvlr15:10:12

when was the last time anybody ran bb on a terminal that doesn’t support colorized output?

borkdude15:10:39

@U5H74UNSF This affects you when you're piping output to another tool, which is quite common

borkdude15:10:27

I have yet to test this with the new 22.3.0, but code like this won't work reliable: https://github.com/AvisoNovate/pretty/blob/716bf0647ab92e5bfce629f4919289e2790c2704/src/io/aviso/ansi.clj#L36

borkdude15:10:31

The difference:

$ bb -e '(binding [*out* *err*] (prn (if (System/console) :console :pipe)))' > /tmp/foo.clj
:pipe

$ bb -e '(binding [*out* *err*] (prn (if (System/console) :console :pipe)))'
:console

mkvlr15:10:02

could it be patched default to color? If you can still turn it off via an ENV that seems fine?

mkvlr15:10:55

holding back loom for compatibility with PDB-11s seems silly to me 😛 unless there’s an aspect that matters today that I’m missing

mkvlr15:10:06

and the custom build option isn’t with the effort imo

mkvlr15:10:34

doubt we’d see any usage based on that

mkvlr15:10:57

lastly subject to change in the JDK has about the same meaning as alpha in clojure: none

borkdude15:10:45

not all code has the option to disable color output with an env var. bb takes 1.5 minutes to build on an arm mac. if people desperately want to try the virtual threads with bb now they can do it

mkvlr15:10:45

sure, the same can be said of uncolorized output however

borkdude15:10:54

the issue with colorized output is that you will get mangled output in a piped to file which can cause bugs

mkvlr15:10:43

could bb include a flag to filter ansi escape sequences?

borkdude15:10:57

@U5H74UNSF Not sure if it can reliably do that, also a performance concern, and I'd rather not add any flags that should be removed afterwards. What would be your main use case for bb + jdk19 virtual threads and have you tried the previous binaries I linked last time?

lread15:10:36

I'd still be very wary of adopting jdk preview features in bb. But... for the color thing, https://no-color.org/?

borkdude15:10:08

That env var should be supported by libraries, not a platform. Bb itself doesn't output any ansi codes anywhere

👍 1
borkdude15:10:43

jet does do this and it uses a more sophisticated way of checking terminal connected-ness, since Console returns null both when you pipe input and output, but only the output matters

borkdude16:10:23

22.3.0 indeed seems to manifest this bug with jdk19:

$ bb -e '(binding [*out* *err*] (prn (if (System/console) :console :pipe)))'
:console

$ bb -e '(binding [*out* *err*] (prn (if (System/console) :console :pipe)))' > /tmp/foo.clj
:console

mkvlr17:10:21

@U04V15CAJ im eager to move away from core.async (which is hard to reason about and ruins stack traces). This isn’t exclusive to bb of course but knowing the code is compatible here is a plus as we’re now targeting bb for our more shortlived http deployments.

mkvlr17:10:29

I still miss the reliability of the erlang VM after switching to clojure so I’m very excited about loom

lread17:10:58

I suppose bb could have an --enable-preview like the Java folks have?

borkdude18:10:10

You can't really switch the JVM's version with which bb is built with a command line flag though?

lread18:10:35

No, but I suppose you could throw if someone used a JDK19 feature/class. Could be too much of a pita to manage?

borkdude18:10:16

curl -sLO 
tar xzvf babashka-1.0.165-SNAPSHOT-macos-aarch64.tar.gz
./bb
That one should work for arm64 macOS ^ The rest of the OSes can download from #CSDUA8S6B if you want to try it out

borkdude18:10:29

The flag is not worth the hassle. I'm not concerned about people using classes that will change, I think early adopters of virtual threads will be aware or at least they should

👍 2
borkdude18:10:28

It's the System/console edge case that bugs me. It seems that the one built on cirrus always returns nil:

$ ./bb -e '(prn (System/console))'
nil

borkdude18:10:27

I'll ask in the Graal slack if there is a plan to release a follow up update

borkdude18:10:45

This is a program that should work with the above binary:

(import '[java.util.concurrent Executors Future])

(defn loads-of-tasks
  [concurrency]
  (let [executor   (Executors/newVirtualThreadPerTaskExecutor)
        ;; executor   (Executors/newFixedThreadPool concurrency)
        tasks      (mapv #(fn []
                            (Thread/sleep 1000)
                            %)
                         (range concurrency))
        start-time (System/currentTimeMillis)
        sum        (->> (.invokeAll executor tasks)
                        (map #(.get ^Future %))
                        (reduce +))
        end-time   (System/currentTimeMillis)]
    (println "Blazingly Fast!")
    {:sum     sum
     :time-ms (- end-time start-time)}))

(loads-of-tasks 100000)

borkdude18:10:55

> A fix for this is already on master and ships with dev builds

pizzaspin 3
mkvlr18:10:50

so soon we can have our virtual threads on consoles?

borkdude18:10:46

not sure when they will release another one

borkdude18:10:28

perhaps I can depend on a dev build - but it's not clear to me how "dev" those are

mkvlr18:10:40

I’m personally fine with waiting on the next graal release, more so than waiting on the JDK to declare the feature stable in JDK 25+

borkdude18:10:09

at least there is a branch that I can update regularly for people who want to try it out right now and also pre-built binaries I can provide

👍 1
borkdude19:10:03

I also noticed one failing build on Windows with JDK 19

borkdude19:10:09

something with hmac256

borkdude19:10:05

I think I have a fix for that, probably an encoding issue on Windows in my test

Crispin02:10:21

I vote to wait for LTS before adopting. Having a jdk19 bb build available for the frontier explorers enables them to discover things that may be an issue before we affect everyday users.

👍 1
borkdude19:02:08

The console bug seems to be solved in 22.3.1:

$ ./bb -e '(prn (System/console))'
#object[java.io.Console 0x42e8df80 "java.io.Console@42e8df80"]
(with the updated jdk19-loom branch on cirrus CI)

borkdude18:10:06

If anyone is interested in trying out bb with jdk 19 virtual threads You should be able to run this program with 100k threads:

(import '[java.util.concurrent Executors Future])

(defn loads-of-tasks
  [concurrency]
  (let [executor   (Executors/newVirtualThreadPerTaskExecutor)
        ;; executor   (Executors/newFixedThreadPool concurrency)
        tasks      (mapv #(fn []
                            (Thread/sleep 1000)
                            %)
                         (range concurrency))
        start-time (System/currentTimeMillis)
        sum        (->> (.invokeAll executor tasks)
                        (map #(.get ^Future %))
                        (reduce +))
        end-time   (System/currentTimeMillis)]
    (println "Blazingly Fast!"
             {:sum     sum
              :time-ms (- end-time start-time)})))

(loads-of-tasks 100000)
Check #babashka-circleci-builds for the jdk19-loom branch builds. For arm64 mac, you can use https://api.cirrus-ci.com/v1/artifact/task/6454117149704192/binaries/babashka-1.0.165-SNAPSHOT-macos-aarch64.tar.gz Or build yourself using native-image 22.3.1 jdk 19.

🙋 2
4
🎸 1
🆒 1
steveb8n22:10:03

Q: lazy question. is there a way to make a cli fn more generic than this? https://github.com/nextdoc/salesforce-babashka/blob/master/src/tasks.clj#L19

steveb8n22:10:54

I’d like to re-use the parse fn with different specs. can this be done somehow? maybe a higher order fn flavour? this seems to be specific to the way the exec fn calls the parse fn so it’s not obvious to me

steveb8n23:10:51

btw I’m writing blog post and making video today. pitching bb to a whole new audience. hopefully it’ll get you a few new sponsors

steveb8n08:10:31

yes it does. and yes, it was a lazy question. now that I checked the CLI source I can see the parse-opts and parse-args fns that will do this for me

👍 1
borkdude08:10:29

looking forward to your blog :) and thanks for those sponsor links

steveb8n08:10:47

let me know if you get any sponsors from this community

borkdude08:10:39

cool! I notice a minor typo: Bashashka

borkdude08:10:37

nice work! will you share this to #C8NUSGWG6, reddit etc? :)

steveb8n08:10:31

thanks. typo fixed

steveb8n08:10:38

I’ll share to articles now…