Fork me on GitHub
#babashka-sci-dev
<
2021-12-15
>
lispyclouds09:12:27

@borkdude what is https://github.com/babashka/babashka/pull/1099/files#r769331216 used for? should i comment it back again? apart from this the PR looks quite ready for action 😄

borkdude09:12:06

yeah we can probably leave that out. it's a test that you can run bb with deps.edn

lispyclouds15:12:04

finally! thanks a lot @cap10morgan 🎉

👍 2
borkdude15:12:51

nice! so now you can run these images on arm cloud things - is that the main benefit?

borkdude15:12:21

Please also update the CHANGELOG.md (with a [skip-ci] in the message)

borkdude15:12:37

and also on Mac M1 + Docker I guess

cap10morgan15:12:51

yep, both of those are my use cases :)

1
borkdude15:12:13

I wonder if the docker arm images are way faster on M1

cap10morgan15:12:43

they should be. the amd64 ones were run on qemu

borkdude15:12:05

ooh ouch, that's slow right? can't you "just" run docker via rosetta2 and then run amd64?

borkdude15:12:25

I don't have an M1 yet

borkdude15:12:34

Just had my i9 macbook pro repaired ;)

cap10morgan16:12:49

no, rosetta2 can't run x86_64 virtual machines

cap10morgan16:12:19

so docker's embedded linux VM has to be arm64

borkdude16:12:01

oh, that's really limiting I guess

borkdude16:12:14

at least for early adopters of M1

cap10morgan16:12:53

it's been surprisingly not so far. I thought the same thing going in, but then all of the earlier-than-me adopters were still just getting work done so I took the plunge. been great so far!

cap10morgan16:12:18

but yeah, there was likely more pain earlier than I jumped on board

borkdude16:12:32

I might buy myself a Macbook Air M1 so I can just have the M1 within my reach for testing

borkdude16:12:17

and I can hook it up with a self-hosted circleci / github action runner when releasing 😂

cap10morgan16:12:29

if you want I can show you a nifty thing I recently figured out where you can tell your local docker "always send builds / containers of this arch over to this other host" which you could probably get pretty far w/ a raspberry pi 4 w/ 8GB of RAM. maybe not for graalvm builds, but maybe... 😉

borkdude16:12:04

we already have arm64 for linux, it's only mac which is missing

borkdude16:12:29

I'll get back on your offer when I need it

cap10morgan16:12:05

yeah, it's just nice that the builds and containers are much faster when they don't have to run under qemu

borkdude16:12:05

@cap10morgan curious, can you test how fast this is in m1 arm64 docker?

$ bb -e "(time (loop [val 0 cnt 10000000] (if (pos? cnt) (recur (inc val) (dec cnt)) val)))"
"Elapsed time: 2496.274002 msecs"
10000000

borkdude16:12:30

compared to your rosetta2 usage on the same computer

cap10morgan16:12:32

arm64: "Elapsed time: 1421.087168 msecs" emulated amd64: "Elapsed time: 20665.796677 msecs"

cap10morgan16:12:50

always good to get an order of magnitude speed up 🙂

borkdude16:12:59

the emulated is pretty slow it seems

cap10morgan16:12:13

graalvm native-image builds just grind to a halt

cap10morgan16:12:17

and some things just crash

borkdude16:12:32

wow I feel lucky I haven't made the leap that fast then yet

cap10morgan16:12:09

but it's nice to have b/c it usually works, eventually haha

cap10morgan16:12:09

oh man, it's a tradeoff I'd make again in a heartbeat

cap10morgan16:12:10

everything else is insanely good & fast

borkdude16:12:24

yes, but building native images locally is a thing I do a lot

cap10morgan16:12:28

and since I learned how to farm out docker builds and runs to amd64 hosts, this isn't even much of an issue anymore either

cap10morgan16:12:06

yeah; the graalvm folks are saying early 2022 for darwin/arm64 support

borkdude16:12:23

I'm subscribed to the issue yeah. By that time, spring, I'll have another look.

👍 1
borkdude16:12:41

Perhaps some generous sponsor wants M1 support so bad they'll give me one... 🤞

cap10morgan16:12:51

that would be awesome!

borkdude16:12:53

but by that time CircleCI may also have support for it

borkdude16:12:10

or Github Actions

borkdude16:12:15

whatever comes first

cap10morgan16:12:23

yeah, I think once apple releases a more "pro" arm mac mini a lot of those dominoes will fall

borkdude16:12:30

AWS already has support for it right?

borkdude16:12:55

they just have a giant stack of those mac minis somewhere?

borkdude16:12:16

that's almost 10x slower than my i9 mac

cap10morgan16:12:32

sorry, meant to paste the link not the time again haha

cap10morgan16:12:40

yeah it's not a speed demon

cap10morgan16:12:56

I guess it's running qemu under an arm linux VM 😬

borkdude16:12:57

I noticed that most bb scripts don't need a lot of performance, luckily

borkdude16:12:28

wait, you didn't test test with rosetta2 directly - I meant, no docker

borkdude16:12:40

just straight from your shell

borkdude16:12:53

that shouldn't involve qemu right

cap10morgan16:12:54

you said, "curious, can you test how fast this is in m1 arm64 docker?" 🙂

cap10morgan16:12:06

docker-free test coming right up!

borkdude16:12:39

I said compared to your rosetta2 usage on the same computer but apparently this can be interpreted in multiple ways :)

cap10morgan16:12:39

ah, yeah. no rosetta2 in docker. 🙂

borkdude16:12:30

yeah, I meant: run it in your shell, as you would normally do as a mac user who downloads bb

cap10morgan16:12:31

yep. interestingly 0.7.1-SNAPSHOT in docker appears to be much faster than 0.7.0 outside it. did some perf improvements land?

borkdude16:12:13

no, but I expect that arm has some perf benefits over intel with respect to these kinds of loops

cap10morgan16:12:37

this is all arm64

borkdude16:12:51

sure, but rosetta2 is in between

cap10morgan16:12:57

linux/arm64 inside docker vs. darwin/arm64 outside it (maybe that's the difference)

cap10morgan16:12:02

no, no rosetta2 here

cap10morgan16:12:08

I haven't run the intel one yet

borkdude16:12:15

let me start over, forget everything before

borkdude16:12:24

as a user who installs babashka via brew on m1

borkdude16:12:28

and runs it in your shell, no docker

borkdude16:12:31

how fast is the loop

cap10morgan16:12:34

yes, I understand what you want me to do

cap10morgan16:12:41

I'm pointing out an interesting aside

cap10morgan16:12:08

bb 0.7.1-SNAPSHOT linux/arm64 in docker runs that loop much faster than bb 0.7.0 darwin/arm64 does

cap10morgan16:12:13

just thought that was interesting

borkdude16:12:51

I would be disappointed if that wasn't the case :)

cap10morgan16:12:03

all arm64 above, not amd64 as I unfortunately first typed

cap10morgan16:12:11

so that's what is interesting

cap10morgan16:12:21

no rosetta2 nor emulation at work here

borkdude16:12:27

but how fast is it then, don't keep me in suspense :)

cap10morgan16:12:47

bb 0.7.0 darwin/arm64: "Elapsed time: 2273.238791 msecs"

borkdude16:12:04

ok, I expected this

cap10morgan16:12:19

and then I'll need to download some things to try out intel on rosetta2

borkdude16:12:30

there must be rosetta2 here right, since native graalvm on arm64 isn't a thing yet for mac

cap10morgan16:12:07

ohhhhhhh yes. you're right. I keep forgetting that even though we just talked about it. facepalm

cap10morgan16:12:32

the linux vs. darwin / amd64 vs. arm64 quadrant keeps tripping me up hahaha

borkdude16:12:40

ok, so rosetta2: 2274, docker linux/arm64: 1500, yeah, no surprise there

cap10morgan16:12:53

that makes a ton more sense 🙂

borkdude16:12:58

and then rosetta2 suddenly look that bad anymore

borkdude16:12:08

instead of the ridiculous 20000 number I saw first

borkdude16:12:18

which was apparently qemu

borkdude16:12:37

but then graalvm should also be reasonable under rosetta2 I suspect

cap10morgan16:12:02

yes, it works decently well there

borkdude16:12:22

thanks!

👍 1
cap10morgan16:12:12

also docker is working on direct darwin platform support too: https://github.com/containerd/containerd/pull/4526

cap10morgan16:12:18

that will be nice someday

borkdude16:12:58

so, images that run darwin?

borkdude16:12:27

is that was allows this?

cap10morgan16:12:45

yeah, VM-less Docker on Mac

cap10morgan16:12:02

similar to the windows/amd64 images I guess

borkdude16:12:37

could be a pragmatic alternative to nix perhaps ;)

borkdude16:12:34

or at least a more performant alternative to linux image on mac