Clojurians Log v2

@borkdude do you remember why are we doing https://github.com/babashka/pod-babashka-fswatcher/issues/12#issuecomment-1119306799 ? call WriteInvokeResponse and not WriteErrorResponse? Its been a while for me and I'm failing to recollect 😛 Also since its not reproducible, should i look into it?

@rahul080327 The way I would fix this is based on tests. Write a test that reproduces the error, fix it, while ensuring the other tests also works. The exact details: dunno :)

yeah the person who reported this cant seem to come up with it anymore after a machine restart it seems

I'll ask for a repro then think about it

yes, let's not fix anything without a repro

Are Actions on sci upto date?

Probably not

okay looking into it now

whats the node version needed for node tests?

I would say 14 LTS or so

@rahul080327 It seems to be the macOS build that wasn't running. I'm not sure if it's just that

yeah adding in the matrix to the native one now

The thing that we need CircleCI specifically is: • more memory (8gb. only important for babashka) • aarch64 builds (important for babashka, pods and clj-kondo) macOS builds we can do on Github Actions too (although it would be way slower) Windows we've always done on Appveyor (which in hindsight is good to spread the risk)

yeah afaik action runners are 8G too, not sure about aarch64

actions runners are 6.5gb last time I looked

aarch64 isn't supported

ah 7gb I see now

this is the current one

Hardware specification for Windows and Linux virtual machines:

    2-core CPU
    7 GB of RAM memory
    14 GB of SSD disk space

Hardware specification for macOS virtual machines:

    3-core CPU
    14 GB of RAM memory
    14 GB of SSD disk space

should be enough for bb I think

is aarch64 blocked on circle too?

no, at least not yet... I'm not sure if macos was just blocked because of macos or because it was the last job to run

yeah I'll just catch the actions yamls up, be prepped

clojure-lsp produces aarch64 via qemu, which is hell slow, the job takes more than an hour, which isn't something I would want

could also build manually on m1 macos inside docker, but also something I want to avoid

could also run self-hosted Github actions builds inside my PC maybe, but then I'd have my PC on all day ;)

yeah the qemu path could be really slow

or we could run it on AWS ourselves maybe

anyway, I still hope circleci will fix this :)

yeah this is just my paranoid-infra-goes-to-shit mind kicking in

it's good to have redundancy :)

test-self-hosted in actions = plank in circle?

yes

is there an actions thing for self-hosted?

already I mean

yeah im seeing a test-self-hosted job there

I don't remember ever updating an actions yaml for SCI

its seen more updates than bb at least, going by commits 😛

o wait, I did

I did this while CircleCI had an outage, while I was waiting at the doctors :)

lol

heh, nice

Oh, bb tasks are really paying off here, since it's the same thing no matter where you run it

so I guess for SCI it's already fixed then right?

yeah its pretty nice

yeah im just adding the jdk 11, clj cli, node tooling etc

and the native matrix

why add stuff if it already works?

we don't need the clj cli

so im planning to use the:

- name: Install clojure tools
        uses: DeLaGuardo/setup-clojure@5.0
        with:
          cli: 1.10.1.763
          lein: 2.9.3
          bb: 0.8.2

reduce some extra installs

and:

- name: Prepare java
        uses: actions/setup-java@v2
        with:
          distribution: 'adopt-hotspot'
          java-version: '11'

can leave it out if you want, just do the native matrix on mac and ubuntu?

was installing the clojure cli as circle had it too

yeah, it's fine, just proceed with what you were doing

would be more than happy to drop stuff 😄

we also have a yaml for babashka, but that one is probably grossly out of date

yeah will get there

thanks :)

so clojure cli not needed?

I thought it wasn't needed because bb tasks + (clojure ...) but I see it's still calling script/test/jvm which calls the clojure CLI. Let's changre as little as possible to all of this and just focus on getting it to work

and since it already works, maybe focus on bb first

almost done here, raise the PR and will go there

🚀

ah the the deploy step wont work

we don't need that yet

I'd rather not have 2 CIs deploying

yeah, will leave a TODO

actions bits look good on https://github.com/babashka/sci/pull/722

Merged!

Talking to https://twitter.com/IAmJerdog in DM now, he's looking into it!

🤞🏼

Btw, the "quit job if only .md file has changed" has been on my list for ages now too

that would save credits too

as demanding people to write "skip ci" isn't working

or I could disable PR builds but that's not optimal either

ive used something like https://github.com/tj-actions/changed-files for actions

big monorepos at work

this with the if: ... directive

can have a bb script for that? 😄

of course :)

yeah more bootstrapping test for the bb repo

Maybe we could just run that code using clojure itself when bb hasn't bootstrapped yet

yeah i think all these and maybe the compile could be in tools.build somehow?

Eventually, but we don't need to make that change now

yep

22.1.0 is the correct graal version for bb right?

correct, newest

ah right, appveyor had 6gb. bb barely builds there, it spends a lot of time in GC ;). the only difference with github is that github has 1gb extra. On Circle we have 8gb + 4 CPU instead of 2 on Github

yeah its not the best but more as a backup

maybe we can get some credits on Exoscale and I can have a build cluster running there 😛

BobCD maybe

clojure all the way

I do have credits for exoscale btw

we can have a think if that can be used

I think I'll have to re-request credits every time though

I might be porting my personal VPS to them, but it's only 100 EUR a year or so at my current provider, so I wasn't really in a hurry to do so

I could also just start paying for CircleCI, I think it's somewhere between 300-400 dollars a month

as long as I got enough sponsors ... and if not, I should charge money for the binaries or so

yeah theres ways around it, maybe similar to clojars, we can get some hosted infra for things

right, could move all the lower resource stuff to github actions and only run the critical things on circle

e.g. SCI is low resource

nbb too

clj-kondo probably too and we don't ship aarch64 binaries for that

yeah once actions is at parity, i can think of dividing up the the things

so it's really only bb + pods

or graal finally lands cross compiling 😛

won't happen

yeah, also building in M1 docker isnt too bad, could be used as a fallback

right

but yeah same story on actions, no M1

we don't have m1 builds now anyway

oh I get what you said now, yes, we could use m1 for linux aarch64 in docker

yeah

maybe the aws pod could be deprecated in favor of awyeah

eventually

is .circleci/script/release safe to run on actions too?

I think so yes. I put it there so users didn't execute it

cool

turning actions back on

tried pulling some tricks with the matrix since the code was so out of date, lets see if it all works. should be a big reduction in yaml if it does

actions doesnt seem to be triggered

ah push only on master

its been a while 😄

will look tomorrow

yeah probably some more fixing needed

yes, last time I enabled github actions on babashka was actually during ClojureD 2020

it was relatively new back then so I wanted to try it out

@rahul080327 the buildx stuff, would that also work on github actions?

yeah theres a setup for that too 😄

the actions yaml is like half the circle one, although no aarch64

maybe you can temporarily disable circleci in your actions PR

as it's wasting credits

also I think it's better if github actions didn't wait for the JVM step to finish

best place would be the workflows at the end to disable circle right?

yeah, just comment out the jobs

right, cancelling the job now too

yeah didnt change any of the workflows in actions, will have a better think

yeah, I remember now, the initial version was contributed by someone who already knew github actions and I think he wanted to save the uberjar for other jobs or so. similar discussion that we had for circleci recently

I think on CircleCI only the macos jobs are affected now though

but this is a good exercise :)

yep

I found a conversation with Marc in which he enabled macOS builds for 2 years on 2020 May 7th... so this explains it

right

actually the macos build are the best candidates to move to github actions since github actions has 14gb of memory there

so if only those were affected, we would still be good

yeah sounds like a good plan, im thinking lets get it to parity and disable parts of it

not too bad on linux

52.3s (17.5% of total time) in 108 GCs | Peak RSS: 5.84GB | CPU load: 1.92
Finished generating 'bb' in 4m 57s.

about 2x as slow as circle

but doable

The xmx is very close to the max here: BABASHKA_XMX: -J-Xmx6500m Could be close to an OOM error if it was bumped even higher

on appveyor it's 5g (of 6g in total)

but as long as it works

yeah lets see if it OOMs

This looks like double work in the static linux one right?

yeah i messed up the condition, will push after this

shouldve been the not of the other but i was oversmart

@rahul080327 we could use the official graalvm github action which has musl support

no idea what

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/./libstdc++.a(cp-demangle.o): in function `d_print_comp_inner':
(.text+0x5a04): undefined reference to `__sprintf_chk'
/usr/bin/ld: (.text+0x6219): undefined reference to `__sprintf_chk'
/usr/bin/ld: (.text+0x6764): undefined reference to `__sprintf_chk'
/usr/bin/ld: (.text+0x6bea): undefined reference to `__sprintf_chk'
/usr/bin/ld: (.text+0x6de2): undefined reference to `__sprintf_chk'

ah yes

https://github.com/marketplace/actions/github-action-for-graalvm

pushed

interesting error: https://github.com/babashka/babashka/runs/6337178092?check_suite_focus=true will continue tomorrow