Fork me on GitHub
#babashka-sci-dev
<
2022-01-13
>
lispyclouds16:01:39

@cap10morgan the condition https://github.com/babashka/babashka/blob/master/.circleci/script/docker#L47 would never be true right? As we set the var externally to linux/amd64,linux/arm64 ?

cap10morgan16:01:16

I’ll look into this shortly

cap10morgan18:01:34

so yeah I think this is a holdover from the first approach of building each platform separately and then uploading a manifest

cap10morgan18:01:47

I'll figure out a fix for it and put up a PR

lispyclouds16:01:17

i think we dont have any alpine images since 0.7.2 😞

cap10morgan16:01:03

Has anyone complained? ;) I ask half in jest as a maintainer of the clojure Docker images who has seriously considered dropping the alpine variants lately.

lispyclouds16:01:45

Well i just stumbled upon it when someone wanted to download alpine images and i could not find the alpine ones for the new versions

lispyclouds16:01:57

last one pushed is 0.7.0-alpine which is the one before our change. alpine would be more useful for bb compared to clojure as mostly they are the final image unlike for clojure where probably the final image is from openjdk or something 😄

cap10morgan16:01:32

Yeah. And bb is nice to minimize all the way :)

lispyclouds16:01:47

Fun fact: we actually had FROM scratch but then it broke someone's jenkins that expected to have cd at least 😛 then we needed curl to be there too

😆 1
cap10morgan18:01:27

Hmm, no arm64 version of pod-babashka-aws, eh? Is that a big change to support?

cap10morgan18:01:04

e.g. does bb need to learn how to resolve platform-specific pods? or would it just be a matter of building and publishing it?

cap10morgan19:01:43

oh never mind, you already answered my question here: https://github.com/babashka/pod-babashka-aws/issues/46

cap10morgan19:01:46

I can work on that PR!

cap10morgan19:01:09

looks like it's almost done anyway

borkdude19:01:20

@cap10morgan There is an open issue for pod-babashka-aws and pretty easy to support, just needs the work to be done

borkdude19:01:41

pods already support the amd64/aarch64 difference

borkdude19:01:52

cool thank you!

👍 1
borkdude19:01:30

@cap10morgan why did we have linux-foo before and now you are matching on linux/foo in PR 1141?

cap10morgan19:01:59

@borkdude Just answered on GitHub but I can copy-paste it here: That should have already been in there, but it comes from the fact that docker platform strings look like `os/arch` while GraalVM and babashka use `os-arch`. There was a time when I was using the hyphenated form in this env var, but that was no longer the case by the time the original PR was merged and I just missed converting this one b/c it was never being run anyway.

cap10morgan19:01:33

You'll notice the other platform values (the env var's default and the list we set it to in .circleci/config.yml) are all os/arch

borkdude19:01:56

ah I see!

👍 1
lispyclouds20:01:51

having a look now

cap10morgan20:01:49

that's weird...

cap10morgan20:01:02

either intermittent or an upstream bug seems like

cap10morgan20:01:16

trying to setup a buildx builder instance

lispyclouds20:01:47

had a failure with the buildx setup too, rerunning fixed it, trying it now

👍 1
borkdude20:01:18

thank you guys!

👍 1
1
lispyclouds20:01:29

shoudl we just add a --push to buildx @cap10morgan?

cap10morgan20:01:38

yeah probably

cap10morgan20:01:41

I can do that

cap10morgan20:01:24

oh, although... that would push snapshots too. do we want that?

lispyclouds20:01:58

but lets parameterize the tagging in the buildx too

lispyclouds20:01:16

based on if its a snapshot or not

lispyclouds20:01:11

yeah its a weird error, better to let buildx push

lispyclouds20:01:58

snapshots => babashka/babashka:alpine others => babashka/babashka:<version>-alpine

borkdude20:01:42

that's not how we did it before

borkdude20:01:56

the unqualified version always refers to the latest stable version

borkdude20:01:06

so I would not change that!

lispyclouds20:01:39

ah yes, the long day is taking a toll on me

cap10morgan20:01:50

ok, yeah, I was just reversing that 🙂

lispyclouds20:01:51

when snapshot is false its just alpine

cap10morgan20:01:58

I'll continue reversing it

borkdude20:01:26

we did push snapshots, but under the snapshot tag or so, let me check

cap10morgan20:01:42

alpine is basically latest for alpine

lispyclouds20:01:45

yes checked now they have a SNAPSHOT in the tag

cap10morgan20:01:55

all of that coming shortly

borkdude20:01:11

yes, we pushed to 0.7.3-SNAPSHOT etc

lispyclouds20:01:12

@cap10morgan if you feel like it have a go at moving them to a bb script too, @borkdude and I discussed that its good enough complexity to boostrap maybe 😄

cap10morgan20:01:01

hehe, tempting!

lispyclouds20:01:05

or even bb tasks

cap10morgan20:01:38

I need to get this arm64 pod-babashka-aws PR together first, but then I might just take a crack at it 🙂

cap10morgan20:01:58

ok that stuff is pushed

lispyclouds21:01:03

no hurries! having the alpine images unblocked is first one!

👍 1
lispyclouds21:01:30

would you open another PR?

cap10morgan21:01:23

yep, once I remember how when GH isn't suggesting it :rolling_on_the_floor_laughing:

lispyclouds21:01:04

i guess you need to rebase?

cap10morgan21:01:09

guess so, one sec

cap10morgan21:01:06

phew, ok. cleaned up

lispyclouds21:01:13

will the tag change after the buildx work? wasnt that what was failing before?

cap10morgan21:01:33

yes b/c it's just one platform

cap10morgan21:01:47

I can make it consistent anyway if you want tho

lispyclouds21:01:43

looks okay to me but what was causing the last error? babashka/babashka:alpine not found one?

cap10morgan21:01:40

oh, d'oh. yeah, b/c it's pushing and not loading now. facepalm yeah, you're right, it needs the full command again.

lispyclouds21:01:01

i feel we should just use good old build here

lispyclouds21:01:16

buildx is overkill here

cap10morgan21:01:33

ehhh, yeah. maybe. it's the future and consistent w/ the rest. might even be what you get anyway at this point in the build script given what comes before.

lispyclouds21:01:41

im fine with the control flow here, just can use build?

cap10morgan21:01:51

it will likely be buildx anyway

lispyclouds21:01:40

i guess currently it doesnt like things built with buildx and the being pushed seperately

lispyclouds21:01:51

maybe have a TODO and we can revisit?

cap10morgan21:01:53

hopefully fixed in latest commit

cap10morgan21:01:06

what would the TODO be?

cap10morgan21:01:22

this works and is likely a change they'll require eventually anyway

lispyclouds21:01:46

yeah this looks better, TODO was using build and revist when docker is happy mixing buildx and builds

lispyclouds21:01:50

but this is better now

👍 1
lispyclouds21:01:41

then do you wanna put the above build in the else? its buidling twice now i guess

lispyclouds21:01:50

when its not a snapshot

lispyclouds21:01:48

or it could be cached away :thinking_face: dunno my head spins at these build layer caching nowadays

borkdude21:01:06

ok, build finished it seems

borkdude21:01:20

do the snapshot images look ok now?

lispyclouds21:01:24

yep looks good! thanks @cap10morgan

cap10morgan21:01:07

so for the pod-babashka-aws aarch64 stuff, circleci only provides arm on machine instances, not docker, huh?

cap10morgan21:01:41

looks like the other builds are docker-based, so there's probably some additional work needed there to setup e.g. localstack

borkdude21:01:22

@cap10morgan we could just skip the localstack stuff for that build

cap10morgan21:01:52

ok, great! that's a big help 🙂

lispyclouds21:01:53

maybe setup_remote_docker could help, run localstack on that?

cap10morgan21:01:01

yeah that's a possibility too

cap10morgan21:01:10

does that work in machine instances? :thinking_face:

lispyclouds21:01:28

i guess so, been a WHILE since ive used it

lispyclouds21:01:05

but looking at the examples maybe not

borkdude21:01:20

just skip it, make the aarch64 compilation work and be done with it

1
👍 1
borkdude21:01:58

there's perhaps one or two calls that don't need AWS, e.g. for fetching the available AWS things it can do, we could just test that as a smoke test