This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-10-25
Channels
- # babashka (65)
- # beginners (34)
- # biff (18)
- # calva (8)
- # clara (22)
- # clj-kondo (32)
- # clojure (24)
- # clojure-bay-area (4)
- # clojure-europe (135)
- # clojure-nl (3)
- # clojure-norway (9)
- # clojure-uk (1)
- # clojurescript (11)
- # clojutre (1)
- # core-async (8)
- # cursive (3)
- # datomic (31)
- # emacs (5)
- # fulcro (6)
- # graalvm (5)
- # graphql (3)
- # honeysql (1)
- # introduce-yourself (9)
- # kaocha (1)
- # lsp (65)
- # meander (5)
- # nbb (7)
- # nrepl (2)
- # off-topic (44)
- # rum (3)
- # shadow-cljs (23)
- # specter (1)
- # tools-deps (6)
- # vim (3)
- # xtdb (30)
Q: where is a good example of github actions using bb? I’m about to add CI to a new OSS project
Have a look how Portal uses bb tasks in GitHub actions — I got some inspiration from that! https://github.com/djblue/portal I like that everything is bundled in BB tasks, and the Github workflow runs those.
@steveb8n setup-clojure is the all-in-one which can also install bb (and clj-kondo): https://github.com/DeLaGuardo/setup-clojure/ A dedicated one: https://github.com/turtlequeue/setup-babashka
Native image 22.3 is out with support for Java 19. This means we can have virtual threads in #babashka - if we move to Java 19 :)
https://www.graalvm.org/release-notes/22_3/#native-image
There is one caveat:
> Note that the java.io.Console
API does not work properly on JDK 19: The information whether or not a Console
is available is wrongly computed at build time.
So now the question is: do we wait for Java next LTS, or do we go with Java 19 for the next bb. Another option is to update the jdk19-loom
branch so you can compile your own bb with virtual thread support.
Follow up in 🧵 .
I would say, let's stay on the stable and let early adopters experiment with the jdk19-loom
branch
The console bug would affect you with tools that e.g. look if you are connected to a console and then print colorized output instead of raw output
when was the last time anybody ran bb on a terminal that doesn’t support colorized output?
@U5H74UNSF This affects you when you're piping output to another tool, which is quite common
I have yet to test this with the new 22.3.0, but code like this won't work reliable: https://github.com/AvisoNovate/pretty/blob/716bf0647ab92e5bfce629f4919289e2790c2704/src/io/aviso/ansi.clj#L36
The difference:
$ bb -e '(binding [*out* *err*] (prn (if (System/console) :console :pipe)))' > /tmp/foo.clj
:pipe
$ bb -e '(binding [*out* *err*] (prn (if (System/console) :console :pipe)))'
:console
could it be patched default to color? If you can still turn it off via an ENV that seems fine?
holding back loom for compatibility with PDB-11s seems silly to me 😛 unless there’s an aspect that matters today that I’m missing
lastly subject to change in the JDK has about the same meaning as alpha in clojure: none
not all code has the option to disable color output with an env var. bb takes 1.5 minutes to build on an arm mac. if people desperately want to try the virtual threads with bb now they can do it
the issue with colorized output is that you will get mangled output in a piped to file which can cause bugs
@U5H74UNSF Not sure if it can reliably do that, also a performance concern, and I'd rather not add any flags that should be removed afterwards. What would be your main use case for bb + jdk19 virtual threads and have you tried the previous binaries I linked last time?
I'd still be very wary of adopting jdk preview features in bb. But... for the color thing, https://no-color.org/?
That env var should be supported by libraries, not a platform. Bb itself doesn't output any ansi codes anywhere
jet
does do this and it uses a more sophisticated way of checking terminal connected-ness, since Console
returns null both when you pipe input and output, but only the output matters
22.3.0 indeed seems to manifest this bug with jdk19:
$ bb -e '(binding [*out* *err*] (prn (if (System/console) :console :pipe)))'
:console
$ bb -e '(binding [*out* *err*] (prn (if (System/console) :console :pipe)))' > /tmp/foo.clj
:console
@U04V15CAJ im eager to move away from core.async (which is hard to reason about and ruins stack traces). This isn’t exclusive to bb of course but knowing the code is compatible here is a plus as we’re now targeting bb for our more shortlived http deployments.
I still miss the reliability of the erlang VM after switching to clojure so I’m very excited about loom
You can't really switch the JVM's version with which bb is built with a command line flag though?
No, but I suppose you could throw if someone used a JDK19 feature/class. Could be too much of a pita to manage?
curl -sLO
tar xzvf babashka-1.0.165-SNAPSHOT-macos-aarch64.tar.gz
./bb
That one should work for arm64 macOS ^
The rest of the OSes can download from #CSDUA8S6B if you want to try it outThe flag is not worth the hassle. I'm not concerned about people using classes that will change, I think early adopters of virtual threads will be aware or at least they should
It's the System/console
edge case that bugs me. It seems that the one built on cirrus always returns nil:
$ ./bb -e '(prn (System/console))'
nil
This is a program that should work with the above binary:
(import '[java.util.concurrent Executors Future])
(defn loads-of-tasks
[concurrency]
(let [executor (Executors/newVirtualThreadPerTaskExecutor)
;; executor (Executors/newFixedThreadPool concurrency)
tasks (mapv #(fn []
(Thread/sleep 1000)
%)
(range concurrency))
start-time (System/currentTimeMillis)
sum (->> (.invokeAll executor tasks)
(map #(.get ^Future %))
(reduce +))
end-time (System/currentTimeMillis)]
(println "Blazingly Fast!")
{:sum sum
:time-ms (- end-time start-time)}))
(loads-of-tasks 100000)
I’m personally fine with waiting on the next graal release, more so than waiting on the JDK to declare the feature stable in JDK 25+
at least there is a branch that I can update regularly for people who want to try it out right now and also pre-built binaries I can provide
I vote to wait for LTS before adopting. Having a jdk19 bb build available for the frontier explorers enables them to discover things that may be an issue before we affect everyday users.
The console bug seems to be solved in 22.3.1:
$ ./bb -e '(prn (System/console))'
#object[java.io.Console 0x42e8df80 "java.io.Console@42e8df80"]
(with the updated jdk19-loom branch on cirrus CI)If anyone is interested in trying out bb with jdk 19 virtual threads You should be able to run this program with 100k threads:
(import '[java.util.concurrent Executors Future])
(defn loads-of-tasks
[concurrency]
(let [executor (Executors/newVirtualThreadPerTaskExecutor)
;; executor (Executors/newFixedThreadPool concurrency)
tasks (mapv #(fn []
(Thread/sleep 1000)
%)
(range concurrency))
start-time (System/currentTimeMillis)
sum (->> (.invokeAll executor tasks)
(map #(.get ^Future %))
(reduce +))
end-time (System/currentTimeMillis)]
(println "Blazingly Fast!"
{:sum sum
:time-ms (- end-time start-time)})))
(loads-of-tasks 100000)
Check #babashka-circleci-builds for the jdk19-loom
branch builds. For arm64 mac, you can use https://api.cirrus-ci.com/v1/artifact/task/6454117149704192/binaries/babashka-1.0.165-SNAPSHOT-macos-aarch64.tar.gz
Or build yourself using native-image 22.3.1 jdk 19.Q: lazy question. is there a way to make a cli fn more generic than this? https://github.com/nextdoc/salesforce-babashka/blob/master/src/tasks.clj#L19
I’d like to re-use the parse fn with different specs. can this be done somehow? maybe a higher order fn flavour? this seems to be specific to the way the exec fn calls the parse fn so it’s not obvious to me
btw I’m writing blog post and making video today. pitching bb to a whole new audience. hopefully it’ll get you a few new sponsors
Maybe this is useful? https://blog.michielborkent.nl/babashka-tasks-meets-babashka-cli.html
yes it does. and yes, it was a lazy question. now that I checked the CLI source I can see the parse-opts and parse-args fns that will do this for me