babashka-sci-dev

lispyclouds 2021-12-15T09:02:27.107900Z

@borkdude what is https://github.com/babashka/babashka/pull/1099/files#r769331216 used for? should i comment it back again? apart from this the PR looks quite ready for action 😄

borkdude 2021-12-15T09:17:06.108300Z

yeah we can probably leave that out. it's a test that you can run bb with deps.edn

lispyclouds 2021-12-15T15:45:04.108700Z

finally! thanks a lot @cap10morgan 🎉

👍 2
borkdude 2021-12-15T15:50:51.109500Z

nice! so now you can run these images on arm cloud things - is that the main benefit?

borkdude 2021-12-15T15:51:21.110Z

Please also update the CHANGELOG.md (with a [skip-ci] in the message)

borkdude 2021-12-15T15:54:37.110200Z

and also on Mac M1 + Docker I guess

cap10morgan 2021-12-15T15:55:51.110600Z

yep, both of those are my use cases :)

➕ 1
borkdude 2021-12-15T15:56:13.111Z

I wonder if the docker arm images are way faster on M1

cap10morgan 2021-12-15T15:56:43.111300Z

they should be. the amd64 ones were run on qemu

borkdude 2021-12-15T15:58:05.112100Z

ooh ouch, that's slow right? can't you "just" run docker via rosetta2 and then run amd64?

borkdude 2021-12-15T15:59:25.112300Z

I don't have an M1 yet

borkdude 2021-12-15T15:59:34.112600Z

Just had my i9 macbook pro repaired ;)

cap10morgan 2021-12-15T16:06:49.113100Z

no, rosetta2 can't run x86_64 virtual machines

cap10morgan 2021-12-15T16:07:19.113400Z

so docker's embedded linux VM has to be arm64

borkdude 2021-12-15T16:08:01.113600Z

oh, that's really limiting I guess

borkdude 2021-12-15T16:08:14.113800Z

at least for early adopters of M1

cap10morgan 2021-12-15T16:08:53.114500Z

it's been surprisingly not so far. I thought the same thing going in, but then all of the earlier-than-me adopters were still just getting work done so I took the plunge. been great so far!

cap10morgan 2021-12-15T16:09:18.115Z

but yeah, there was likely more pain earlier than I jumped on board

borkdude 2021-12-15T16:09:32.115300Z

I might buy myself a Macbook Air M1 so I can just have the M1 within my reach for testing

borkdude 2021-12-15T16:10:17.115900Z

and I can hook it up with a self-hosted circleci / github action runner when releasing 😂

cap10morgan 2021-12-15T16:12:29.116900Z

if you want I can show you a nifty thing I recently figured out where you can tell your local docker "always send builds / containers of this arch over to this other host" which you could probably get pretty far w/ a raspberry pi 4 w/ 8GB of RAM. maybe not for graalvm builds, but maybe... 😉

borkdude 2021-12-15T16:13:04.117300Z

we already have arm64 for linux, it's only mac which is missing

borkdude 2021-12-15T16:13:29.117500Z

I'll get back on your offer when I need it

cap10morgan 2021-12-15T16:40:05.144200Z

yeah, it's just nice that the builds and containers are much faster when they don't have to run under qemu

borkdude 2021-12-15T16:11:05.116400Z

@cap10morgan curious, can you test how fast this is in m1 arm64 docker?

$ bb -e "(time (loop [val 0 cnt 10000000] (if (pos? cnt) (recur (inc val) (dec cnt)) val)))"
"Elapsed time: 2496.274002 msecs"
10000000

borkdude 2021-12-15T16:11:30.116800Z

compared to your rosetta2 usage on the same computer

cap10morgan 2021-12-15T16:12:44.117200Z

yep

cap10morgan 2021-12-15T16:20:32.118200Z

arm64: "Elapsed time: 1421.087168 msecs" emulated amd64: "Elapsed time: 20665.796677 msecs"

borkdude 2021-12-15T16:20:39.118600Z

niiice

cap10morgan 2021-12-15T16:20:50.118800Z

always good to get an order of magnitude speed up 🙂

borkdude 2021-12-15T16:20:59.119Z

the emulated is pretty slow it seems

cap10morgan 2021-12-15T16:21:03.119200Z

yes

cap10morgan 2021-12-15T16:21:13.119500Z

graalvm native-image builds just grind to a halt

cap10morgan 2021-12-15T16:21:17.119700Z

and some things just crash

borkdude 2021-12-15T16:21:32.120200Z

wow I feel lucky I haven't made the leap that fast then yet

cap10morgan 2021-12-15T16:22:09.120900Z

but it's nice to have b/c it usually works, eventually haha

cap10morgan 2021-12-15T16:22:09.121Z

oh man, it's a tradeoff I'd make again in a heartbeat

cap10morgan 2021-12-15T16:22:10.121100Z

everything else is insanely good & fast

borkdude 2021-12-15T16:22:24.121700Z

yes, but building native images locally is a thing I do a lot

cap10morgan 2021-12-15T16:22:28.121800Z

and since I learned how to farm out docker builds and runs to amd64 hosts, this isn't even much of an issue anymore either

cap10morgan 2021-12-15T16:23:06.122300Z

yeah; the graalvm folks are saying early 2022 for darwin/arm64 support

borkdude 2021-12-15T16:23:23.122700Z

I'm subscribed to the issue yeah. By that time, spring, I'll have another look.

👍 1
borkdude 2021-12-15T16:23:41.123200Z

Perhaps some generous sponsor wants M1 support so bad they'll give me one... 🤞

cap10morgan 2021-12-15T16:23:51.123500Z

that would be awesome!

borkdude 2021-12-15T16:23:53.123700Z

but by that time CircleCI may also have support for it

borkdude 2021-12-15T16:24:10.124200Z

or Github Actions

borkdude 2021-12-15T16:24:15.124400Z

whatever comes first

cap10morgan 2021-12-15T16:24:23.124700Z

yeah, I think once apple releases a more "pro" arm mac mini a lot of those dominoes will fall

borkdude 2021-12-15T16:24:30.125Z

AWS already has support for it right?

cap10morgan 2021-12-15T16:24:40.125300Z

yeah

borkdude 2021-12-15T16:24:55.125900Z

they just have a giant stack of those mac minis somewhere?

borkdude 2021-12-15T16:25:16.126400Z

that's almost 10x slower than my i9 mac

cap10morgan 2021-12-15T16:25:32.126700Z

sorry, meant to paste the link not the time again haha

cap10morgan 2021-12-15T16:25:40.127200Z

yeah it's not a speed demon

cap10morgan 2021-12-15T16:25:56.127800Z

I guess it's running qemu under an arm linux VM 😬

borkdude 2021-12-15T16:25:57.127900Z

I noticed that most bb scripts don't need a lot of performance, luckily

cap10morgan 2021-12-15T16:26:09.128200Z

yep!

borkdude 2021-12-15T16:26:28.128600Z

wait, you didn't test test with rosetta2 directly - I meant, no docker

borkdude 2021-12-15T16:26:40.128800Z

just straight from your shell

borkdude 2021-12-15T16:26:53.129100Z

that shouldn't involve qemu right

cap10morgan 2021-12-15T16:26:54.129200Z

you said, "curious, can you test how fast this is in m1 arm64 docker?" 🙂

cap10morgan 2021-12-15T16:27:06.129500Z

docker-free test coming right up!

borkdude 2021-12-15T16:27:39.130Z

I said compared to your rosetta2 usage on the same computer but apparently this can be interpreted in multiple ways :)

cap10morgan 2021-12-15T16:28:39.130300Z

ah, yeah. no rosetta2 in docker. 🙂

borkdude 2021-12-15T16:29:30.130700Z

yeah, I meant: run it in your shell, as you would normally do as a mac user who downloads bb

cap10morgan 2021-12-15T16:30:31.131600Z

yep. interestingly 0.7.1-SNAPSHOT in docker appears to be much faster than 0.7.0 outside it. did some perf improvements land?

borkdude 2021-12-15T16:31:13.132300Z

no, but I expect that arm has some perf benefits over intel with respect to these kinds of loops

cap10morgan 2021-12-15T16:31:37.132800Z

this is all arm64

borkdude 2021-12-15T16:31:51.133300Z

sure, but rosetta2 is in between

cap10morgan 2021-12-15T16:31:57.133600Z

linux/arm64 inside docker vs. darwin/arm64 outside it (maybe that's the difference)

cap10morgan 2021-12-15T16:32:02.133800Z

no, no rosetta2 here

cap10morgan 2021-12-15T16:32:08.134100Z

I haven't run the intel one yet

borkdude 2021-12-15T16:32:15.134300Z

let me start over, forget everything before

borkdude 2021-12-15T16:32:24.134600Z

as a user who installs babashka via brew on m1

borkdude 2021-12-15T16:32:28.134900Z

and runs it in your shell, no docker

borkdude 2021-12-15T16:32:31.135100Z

how fast is the loop

cap10morgan 2021-12-15T16:32:34.135300Z

yes, I understand what you want me to do

cap10morgan 2021-12-15T16:32:41.135600Z

I'm pointing out an interesting aside

cap10morgan 2021-12-15T16:33:08.136200Z

bb 0.7.1-SNAPSHOT linux/arm64 in docker runs that loop much faster than bb 0.7.0 darwin/arm64 does

cap10morgan 2021-12-15T16:33:13.136400Z

just thought that was interesting

borkdude 2021-12-15T16:33:18.136600Z

yeah

borkdude 2021-12-15T16:33:19.136800Z

cool

borkdude 2021-12-15T16:33:51.137600Z

I would be disappointed if that wasn't the case :)

cap10morgan 2021-12-15T16:34:03.137900Z

all arm64 above, not amd64 as I unfortunately first typed

cap10morgan 2021-12-15T16:34:11.138100Z

so that's what is interesting

cap10morgan 2021-12-15T16:34:21.138500Z

no rosetta2 nor emulation at work here

borkdude 2021-12-15T16:34:27.138700Z

but how fast is it then, don't keep me in suspense :)

cap10morgan 2021-12-15T16:34:47.139Z

bb 0.7.0 darwin/arm64: "Elapsed time: 2273.238791 msecs"

borkdude 2021-12-15T16:35:04.139400Z

ok, I expected this

cap10morgan 2021-12-15T16:35:19.140Z

and then I'll need to download some things to try out intel on rosetta2

borkdude 2021-12-15T16:35:30.140300Z

there must be rosetta2 here right, since native graalvm on arm64 isn't a thing yet for mac

cap10morgan 2021-12-15T16:36:07.140800Z

ohhhhhhh yes. you're right. I keep forgetting that even though we just talked about it. facepalm

cap10morgan 2021-12-15T16:36:32.141700Z

the linux vs. darwin / amd64 vs. arm64 quadrant keeps tripping me up hahaha

borkdude 2021-12-15T16:36:40.141900Z

ok, so rosetta2: 2274, docker linux/arm64: 1500, yeah, no surprise there

cap10morgan 2021-12-15T16:36:53.142200Z

that makes a ton more sense 🙂

borkdude 2021-12-15T16:36:58.142400Z

and then rosetta2 suddenly look that bad anymore

borkdude 2021-12-15T16:37:08.142700Z

instead of the ridiculous 20000 number I saw first

borkdude 2021-12-15T16:37:18.142900Z

which was apparently qemu

cap10morgan 2021-12-15T16:37:32.143200Z

yes

borkdude 2021-12-15T16:37:37.143400Z

but then graalvm should also be reasonable under rosetta2 I suspect

cap10morgan 2021-12-15T16:38:02.143800Z

yes, it works decently well there

borkdude 2021-12-15T16:38:22.144Z

thanks!

👍 1
cap10morgan 2021-12-15T16:41:12.144700Z

also docker is working on direct darwin platform support too: https://github.com/containerd/containerd/pull/4526

cap10morgan 2021-12-15T16:41:18.145Z

that will be nice someday

borkdude 2021-12-15T16:41:58.145200Z

so, images that run darwin?

borkdude 2021-12-15T16:45:27.145600Z

is that was allows this?

cap10morgan 2021-12-15T16:45:45.145800Z

yeah, VM-less Docker on Mac

cap10morgan 2021-12-15T16:46:02.146100Z

similar to the windows/amd64 images I guess

borkdude 2021-12-15T16:47:37.146400Z

could be a pragmatic alternative to nix perhaps ;)

borkdude 2021-12-15T16:49:34.146800Z

or at least a more performant alternative to linux image on mac

cap10morgan 2021-12-15T16:49:50.147Z

yeah