babashka-sci-dev

lispyclouds 2022-01-13T16:26:39.000900Z

@cap10morgan the condition https://github.com/babashka/babashka/blob/master/.circleci/script/docker#L47 would never be true right? As we set the var externally to linux/amd64,linux/arm64 ?

cap10morgan 2022-01-13T16:47:16.003500Z

I’ll look into this shortly

lispyclouds 2022-01-13T16:49:10.003900Z

Thanks!

cap10morgan 2022-01-13T18:14:34.005400Z

so yeah I think this is a holdover from the first approach of building each platform separately and then uploading a manifest

cap10morgan 2022-01-13T18:14:47.005600Z

I'll figure out a fix for it and put up a PR

cap10morgan 2022-01-13T18:28:28.005800Z

think this should do it: https://github.com/babashka/babashka/pull/1141

lispyclouds 2022-01-13T16:27:17.001600Z

i think we dont have any alpine images since 0.7.2 😞

cap10morgan 2022-01-13T16:47:03.003100Z

Has anyone complained? ;) I ask half in jest as a maintainer of the clojure Docker images who has seriously considered dropping the alpine variants lately.

lispyclouds 2022-01-13T16:48:45.003700Z

Well i just stumbled upon it when someone wanted to download alpine images and i could not find the alpine ones for the new versions

lispyclouds 2022-01-13T16:50:57.004100Z

last one pushed is 0.7.0-alpine which is the one before our change. alpine would be more useful for bb compared to clojure as mostly they are the final image unlike for clojure where probably the final image is from openjdk or something πŸ˜„

cap10morgan 2022-01-13T16:54:32.004800Z

Yeah. And bb is nice to minimize all the way :)

lispyclouds 2022-01-13T16:57:47.005Z

Fun fact: we actually had FROM scratch but then it broke someone's jenkins that expected to have cd at least πŸ˜› then we needed curl to be there too

πŸ˜† 1
borkdude 2022-01-13T16:34:01.001900Z

hi!

πŸ‘‹ 1
cap10morgan 2022-01-13T18:58:27.006700Z

Hmm, no arm64 version of pod-babashka-aws, eh? Is that a big change to support?

cap10morgan 2022-01-13T18:59:04.007400Z

e.g. does bb need to learn how to resolve platform-specific pods? or would it just be a matter of building and publishing it?

cap10morgan 2022-01-13T19:00:43.007900Z

oh never mind, you already answered my question here: https://github.com/babashka/pod-babashka-aws/issues/46

cap10morgan 2022-01-13T19:00:46.008100Z

I can work on that PR!

cap10morgan 2022-01-13T19:01:09.008500Z

looks like it's almost done anyway

borkdude 2022-01-13T19:02:20.009Z

@cap10morgan There is an open issue for pod-babashka-aws and pretty easy to support, just needs the work to be done

borkdude 2022-01-13T19:02:41.009300Z

pods already support the amd64/aarch64 difference

borkdude 2022-01-13T19:02:52.009500Z

cool thank you!

πŸ‘ 1
borkdude 2022-01-13T19:07:30.010100Z

@cap10morgan why did we have linux-foo before and now you are matching on linux/foo in PR 1141?

cap10morgan 2022-01-13T19:09:59.010600Z

@borkdude Just answered on GitHub but I can copy-paste it here: That should have already been in there, but it comes from the fact that docker platform strings look likeΒ `os/arch` while GraalVM and babashka useΒ `os-arch`. There was a time when I was using the hyphenated form in this env var, but that was no longer the case by the time the original PR was merged and I just missed converting this one b/c it was never being run anyway.

cap10morgan 2022-01-13T19:10:33.011400Z

You'll notice the other platform values (the env var's default and the list we set it to in .circleci/config.yml) are all os/arch

borkdude 2022-01-13T19:10:56.011600Z

ah I see!

πŸ‘ 1
lispyclouds 2022-01-13T20:42:51.012400Z

having a look now

cap10morgan 2022-01-13T20:44:49.012600Z

that's weird...

cap10morgan 2022-01-13T20:45:02.012900Z

either intermittent or an upstream bug seems like

cap10morgan 2022-01-13T20:45:16.013200Z

trying to setup a buildx builder instance

lispyclouds 2022-01-13T20:45:47.013800Z

had a failure with the buildx setup too, rerunning fixed it, trying it now

πŸ‘ 1
borkdude 2022-01-13T20:46:03.014Z

ok

borkdude 2022-01-13T20:46:18.014600Z

thank you guys!

πŸ‘ 1
πŸ‘πŸΌ 1
lispyclouds 2022-01-13T20:46:29.015Z

shoudl we just add a --push to buildx @cap10morgan?

cap10morgan 2022-01-13T20:46:38.015200Z

yeah probably

cap10morgan 2022-01-13T20:46:41.015400Z

I can do that

cap10morgan 2022-01-13T20:47:24.015900Z

oh, although... that would push snapshots too. do we want that?

lispyclouds 2022-01-13T20:47:37.016100Z

yes

cap10morgan 2022-01-13T20:47:40.016400Z

cool

lispyclouds 2022-01-13T20:47:58.016800Z

but lets parameterize the tagging in the buildx too

lispyclouds 2022-01-13T20:48:16.017100Z

based on if its a snapshot or not

lispyclouds 2022-01-13T20:49:11.017500Z

yeah its a weird error, better to let buildx push

lispyclouds 2022-01-13T20:51:58.018500Z

snapshots => babashka/babashka:alpine others => babashka/babashka:<version>-alpine

borkdude 2022-01-13T20:52:29.019Z

hmm

borkdude 2022-01-13T20:52:42.019200Z

that's not how we did it before

borkdude 2022-01-13T20:52:56.019500Z

the unqualified version always refers to the latest stable version

borkdude 2022-01-13T20:53:06.019800Z

so I would not change that!

lispyclouds 2022-01-13T20:53:39.020100Z

ah yes, the long day is taking a toll on me

cap10morgan 2022-01-13T20:53:50.020700Z

ok, yeah, I was just reversing that πŸ™‚

lispyclouds 2022-01-13T20:53:51.021Z

when snapshot is false its just alpine

cap10morgan 2022-01-13T20:53:58.021200Z

I'll continue reversing it

lispyclouds 2022-01-13T20:54:20.021500Z

thanks for that @borkdude

borkdude 2022-01-13T20:55:26.021900Z

we did push snapshots, but under the snapshot tag or so, let me check

cap10morgan 2022-01-13T20:55:42.022500Z

alpine is basically latest for alpine

lispyclouds 2022-01-13T20:55:45.022700Z

yes checked now they have a SNAPSHOT in the tag

cap10morgan 2022-01-13T20:55:55.022900Z

all of that coming shortly

borkdude 2022-01-13T20:56:11.023100Z

yes, we pushed to 0.7.3-SNAPSHOT etc

lispyclouds 2022-01-13T20:58:12.024400Z

@cap10morgan if you feel like it have a go at moving them to a bb script too, @borkdude and I discussed that its good enough complexity to boostrap maybe πŸ˜„

cap10morgan 2022-01-13T20:59:01.024700Z

hehe, tempting!

lispyclouds 2022-01-13T20:59:05.024800Z

or even bb tasks

cap10morgan 2022-01-13T20:59:38.025400Z

I need to get this arm64 pod-babashka-aws PR together first, but then I might just take a crack at it πŸ™‚

cap10morgan 2022-01-13T20:59:58.025900Z

ok that stuff is pushed

lispyclouds 2022-01-13T21:00:03.026100Z

no hurries! having the alpine images unblocked is first one!

πŸ‘ 1
lispyclouds 2022-01-13T21:00:30.026600Z

would you open another PR?

cap10morgan 2022-01-13T21:04:23.027Z

yep, once I remember how when GH isn't suggesting it 🀣

cap10morgan 2022-01-13T21:04:50.027200Z

found it

lispyclouds 2022-01-13T21:06:04.027600Z

i guess you need to rebase?

cap10morgan 2022-01-13T21:06:09.027800Z

guess so, one sec

cap10morgan 2022-01-13T21:15:06.028500Z

phew, ok. cleaned up

lispyclouds 2022-01-13T21:15:13.028700Z

will the tag change after the buildx work? wasnt that what was failing before?

cap10morgan 2022-01-13T21:15:33.029200Z

yes b/c it's just one platform

cap10morgan 2022-01-13T21:15:47.029700Z

I can make it consistent anyway if you want tho

lispyclouds 2022-01-13T21:16:43.030300Z

looks okay to me but what was causing the last error? babashka/babashka:alpine not found one?

cap10morgan 2022-01-13T21:17:40.030900Z

oh, d'oh. yeah, b/c it's pushing and not loading now. facepalm yeah, you're right, it needs the full command again.

cap10morgan 2022-01-13T21:17:50.031100Z

isn't 100% in love with buildx

lispyclouds 2022-01-13T21:18:01.031400Z

i feel we should just use good old build here

lispyclouds 2022-01-13T21:18:16.031900Z

buildx is overkill here

cap10morgan 2022-01-13T21:19:33.033100Z

ehhh, yeah. maybe. it's the future and consistent w/ the rest. might even be what you get anyway at this point in the build script given what comes before.

lispyclouds 2022-01-13T21:19:41.033200Z

im fine with the control flow here, just can use build?

cap10morgan 2022-01-13T21:19:51.033400Z

it will likely be buildx anyway

lispyclouds 2022-01-13T21:20:40.034100Z

i guess currently it doesnt like things built with buildx and the being pushed seperately

lispyclouds 2022-01-13T21:20:51.034600Z

maybe have a TODO and we can revisit?

cap10morgan 2022-01-13T21:20:53.034700Z

hopefully fixed in latest commit

cap10morgan 2022-01-13T21:21:06.034900Z

what would the TODO be?

cap10morgan 2022-01-13T21:21:22.035500Z

this works and is likely a change they'll require eventually anyway

lispyclouds 2022-01-13T21:21:46.036Z

yeah this looks better, TODO was using build and revist when docker is happy mixing buildx and builds

lispyclouds 2022-01-13T21:21:50.036200Z

but this is better now

πŸ‘ 1
lispyclouds 2022-01-13T21:22:41.036800Z

then do you wanna put the above build in the else? its buidling twice now i guess

lispyclouds 2022-01-13T21:23:50.037100Z

when its not a snapshot

lispyclouds 2022-01-13T21:24:48.037900Z

or it could be cached away πŸ€” dunno my head spins at these build layer caching nowadays

borkdude 2022-01-13T21:49:06.038500Z

ok, build finished it seems

borkdude 2022-01-13T21:49:20.038900Z

do the snapshot images look ok now?

lispyclouds 2022-01-13T21:50:24.039200Z

yep looks good! thanks @cap10morgan

cap10morgan 2022-01-13T21:50:55.039400Z

excellent

cap10morgan 2022-01-13T21:52:07.040200Z

so for the pod-babashka-aws aarch64 stuff, circleci only provides arm on machine instances, not docker, huh?

cap10morgan 2022-01-13T21:52:41.040800Z

looks like the other builds are docker-based, so there's probably some additional work needed there to setup e.g. localstack

borkdude 2022-01-13T21:53:22.041100Z

@cap10morgan we could just skip the localstack stuff for that build

cap10morgan 2022-01-13T21:53:52.041900Z

ok, great! that's a big help πŸ™‚

lispyclouds 2022-01-13T21:53:53.042Z

maybe setup_remote_docker could help, run localstack on that?

cap10morgan 2022-01-13T21:54:01.042200Z

yeah that's a possibility too

cap10morgan 2022-01-13T21:54:10.042500Z

does that work in machine instances? πŸ€”

lispyclouds 2022-01-13T21:54:28.042900Z

i guess so, been a WHILE since ive used it

lispyclouds 2022-01-13T21:55:05.043200Z

but looking at the examples maybe not

borkdude 2022-01-13T21:55:20.043700Z

just skip it, make the aarch64 compilation work and be done with it

πŸ‘ 1
πŸ‘πŸΌ 1
borkdude 2022-01-13T21:56:58.044800Z

there's perhaps one or two calls that don't need AWS, e.g. for fetching the available AWS things it can do, we could just test that as a smoke test