I'll do this in an hour, sorry... again :)

haha its ok!

done!

thank you!

Is this the static binary? And which OS is this?

It runs under ubuntu 20.04 image

right and how is the clj-kondo binary downloaded? is it an script install or something else?

yep, I think we run this

bash <(curl -s )

I'm also getting this in a docker ubuntu image :/ root@f0047079433a:/# clj-kondo qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

what does ldd <clj-kondo-binary> and file <clj-kondo-binary> say?

has graalvm increased base glibc version?

it does work for me when I do:

docker run --rm --platform linux/x86_64  -it ubuntu

@U01PG2LDYV8 Could it be you're running in docker as well?

yeah the above error you got is an arch mismatch @borkdude

yes, running it in docker

on wsl2 I also get:

$ /home/borkdude/.linuxbrew/bin/clj-kondo
/home/borkdude/.linuxbrew/bin/clj-kondo: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/borkdude/.linuxbrew/bin/clj-kondo)
/home/borkdude/.linuxbrew/bin/clj-kondo: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /home/borkdude/.linuxbrew/bin/clj-kondo)

could you tell what ldd clj-kondo says? not at a linux machine atm

$ ldd /home/borkdude/.linuxbrew/bin/clj-kondo
/home/borkdude/.linuxbrew/bin/clj-kondo: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/borkdude/.linuxbrew/bin/clj-kondo)
/home/borkdude/.linuxbrew/bin/clj-kondo: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /home/borkdude/.linuxbrew/bin/clj-kondo)
        linux-vdso.so.1 (0x00007ffc60758000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f0a76c5a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0a76a68000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f0a796e9000)

and ldd --version ?

there should be a glibc somewhere there

$ ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.2) 2.31

well something has increased the base requirement, my suspicison is native-image

newer linux images might work. ubuntu 22 i think

I haven't heard about this problem with babashka

bb works just fine

could it be related to the alpine thing I "fixed" yesterday?

well thats only on the alpine image

yeah, so this was the only change in the circleci build: https://github.com/clj-kondo/clj-kondo/commit/a3c9282e2160ec227c46d8dd71d43c49b4fb9aae

I'm using the same graal version for bb

but this seems to be graal 17 and bb is 11 i guess?

graal 17...?

GRAALVM_HOME: /home/circleci/graalvm-ce-java17-22.3.0 java17

oh, I didn't even realize that

also i see a mix of 11 and 17 in the circle yaml

yeah I never intended to upgrade to 17

fuck

but this should work as well, right

so I'll ask in the graalvm chat

i think so, im just going on the hunch of something bumping the glibc version. reading the release notes

we could try setting it all to java11 as bb?

I see:

GraalVM will no longer run on some older Linux AMD64 distributions. It is now built against glibc 2.17 (instead of 2.12), and as a result it may not run on Oracle Linux 6 or other RHEL 6 derivatives.

in https://www.graalvm.org/release-notes/22_2/

well this is still older than whats here

yeah

I started a discussion here: https://graalvm.slack.com/archives/CN9KSFB40/p1670581560658649

Now I also get why the github action broke

https://github.com/clj-kondo/clj-kondo/actions/runs/3649794985/jobs/6164932624

even though it's not clear from the error message, I'm almost certain it has to do with this

nope java11 didn't help either https://app.circleci.com/pipelines/github/clj-kondo/clj-kondo/5107/workflows/156f979b-bed1-479d-b28f-dafee5d9e054/jobs/21245/steps

perhaps if it detects a too new glibc version on the CI host machine it links against that?

yeah im thinking that too. could you try ldd --version on cimg/clojure:1.11.1-openjdk-17.0

I'm doing this in ssh in circleci now:

$ ldd --version
ldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35

i believe the linux amd64 is in docker? https://github.com/clj-kondo/clj-kondo/blob/master/.circleci/config.yml#L56

On the machine:

$ ldd ./clj-kondo
	linux-vdso.so.1 (0x00007ffcac1d4000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1bc2f19000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1bc2efd000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1bc2cd5000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1bc2bee000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1bc5ba9000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1bc2bce000)

yeah but the error is about 2.32 and 2.34

makes no sense to me

could try a shell in a container of cimg/clojure:1.11.1-openjdk-17.0 and see the ldd version there?

sure

thats where its built in ci

when I ssh into the machine, am I not logging into the running docker img then?

no, its the raw instance

confusing

$ ldd --version
ldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35

looks exactly like the output above (so I think I'm in the same docker img, really)

yeah also the lower kernel is shared so doesnt matter much

Oh, here's a difference, you're right:

$ uname -a
Linux 3320512d8efc 5.15.0-1021-aws #25~20.04.1-Ubuntu SMP Thu Sep 22 13:59:08 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

$ uname -a
Linux b3097fb367af 5.10.76-linuxkit #1 SMP PREEMPT Mon Nov 8 11:22:26 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

ah, could be the container being scheduled on different machines?

anyway, let's focus on the libc thing

I can try to downgrade to the older stuff that bb uses

or just drop the dynamic binary and do static everywhere

except that isn't possible on aarch64

fully static should be better in kondo's case than bb

why so? > + > except that isn't possible on aarch64

yeah the aarch64 issue aside, iirc we do a mostly static build in bb due some issues right?

not fully static

https://github.com/oracle/graal/issues/3737

the issue is just that musl doesn't work for aarch64

right, i guess we are talking about the same thing then

I'll port the install script to download the static binary on linux...

really trying to diff bb's and kondo's build, bb using circleci/clojure:openjdk-11-lein-2.9.8-bullseye vs cimg/clojure:1.11.1-openjdk-17.0 in kondo for docker builds and machines elsewhere. my thoughts are things couldve updated in the cimg base images and the binary links there

the circleci/... images are deprecated and possibly never got updated

i can try to have a deeper look on better machines a bit later in the day. downloading the static one should be a stop gap thing for now i think too

I know they're deprecated

@U01PG2LDYV8 Could you try the install script again? I now tweaked it so it uses the static binary always on linux

At least it works for me now in wsl2

> I know they're deprecated yeah thats one thought in my head how things could break

well, it still works with the deprecated images ;)

I didn't upgrade to cimg myself on clj-kondo, it was a PR

it’s working!

I'll deprecate the non-static image in clj-kondo and babashka on linux in the future except on aarch64, saves maintaining another build too

sounds good to me

LGTM

> I'll deprecate the non-static image in clj-kondo and babashka on linux in the future except on aarch64, saves maintaining another build too Any news about static builds in aarch64-linux?

still same: mostly static except libc

BTW, not related

Probably will try to bump zlib to 1.2.13

There are a few CVEs related to 1.2.11 and 1.2.12, and I think they fixed the issue that caused the errors in 1.2.12

I can have a look at it if you send a PR @UFDRD93RR

Sure

Correct incorrect inputs provided to the CRC functions. This mitigates a bug in Java. 

geez, it really helps to have mac + amd / aarch64 + windows + linux at home for these kinds of things.. I made a mistake on macos + static in the bash script which I could test on my older macbook which is still amd64

so much for Java's Write Once, Run Anywhere

Write Once, Debug Anywhere more likely

that's the issue with distributing binaries

not java's problem, native image problem

yeah was sarcasm 😛

Java does have a few issues with multi-platform though 😛

I am trying to get openjdk inside nixpkgs up-to-date and it is a mess

See "Targeting multiple platforms" paragraph here: https://medium.com/graalvm/babashka-how-graalvm-helped-create-a-fast-starting-scripting-environment-for-clojure-b0fcc38b0746

I'll also update the brew formula to pick the static binary from now on

< Insert rewrite in Rust joke here >

brew formula also updated, works on my ubuntu now

the issue with the musl binaries was that their stack size is tiny except when you run them in a thread (which is then configurable) which caused issues for bb

so when in musl, the whole main program is running inside a "dummy" thread

the overhead for that is probably small enough to not worry about it

Going with this in the bb install script as well:

*) arch=amd64
       # always use static image on linux
       if [[ "$platform" == "linux" ]]; then
           static_binary="true"
       fi
       ;;

I hope that will prevent questions / confusions like these: https://github.com/babashka/babashka/issues/1434

done:

$  bash <(curl -s ) --dir /tmp
Downloading  to /tmp/tmp.yP24TUxaOO
Moving /tmp/bb to /tmp/bb.old
Successfully installed bb in /tmp

Brew bb formula also updated

I wonder if running the whole main program in a thread means bb could use less cores...

probably not right

> I wonder if running the whole main program in a thread means bb could use less cores... It will use extra stack and there is a small overhead. Not really measurable, specially on Linux that is really efficient with Threads

If this was in macOS or Windows (in WSL1 specially), you probably can measure some difference

cool

It seems the Dockerfile has a few issues...

GraalVM only supports building static binaries on x86_64.

this is "our" message ;)

Yeah I know

We basically set BABASHKA_ARCH to amd64 somewhere in the process

That is clearly wrong

@U7ERLH6JX https://github.com/babashka/babashka/pull/1445

CC @borkdude

@UFDRD93RR can you explain why those defaults are removed? it's been a while ... :) cc @U7ERLH6JX

With the defaults I was having the issue above

They're also clearly wrong

Because TARGETARCH for amd64 should not set BABASHKA_ARCH to amd64 too

(`BABASHKA_ARCH` should be x86_64)

Just fyi, I’ll try to have a look in a bit later today. Reaching to better machines and time hopefully 😅

ok @U7ERLH6JX I'll let you review + merge this when you have time

The advice from graalvm slack: build with the oldest ubuntu possible

Damn, even Ubuntu 22 can't run that image: https://github.com/clj-kondo/clj-kondo/issues/1909

I'm doing another clj-kondo release with a docker image that uses the SHA of the image that was used in the previous version

Great that you found the SHA! I'm semi glad too that glibc getting bumped externally hunch played out and we now know where too.

i looked at glibc version on ubuntu:22.04 docker now, its at 2.35. when they move everyone else is moved too i guess

but how quickly :)

Aaand the Github action started working again too https://github.com/clj-kondo/clj-kondo/actions/runs/3663694967/jobs/6193635075

well, annoying but useful learnings. at least for me 😛

I've had this problem before I think once

In graal we had a discussion on being able to set a lower glibc version in the native image despite the build env

hope they will support that, but I guess you'd also have to deal with zlib etc

at least zlib isnt fine grained into the OS like glibc. can control it better

right

I guess glibc is the most difficult one

well if youre google and you solve it by inventing a whole glibc free language to get around it 😅

> on being able to set a lower glibc version in the native image despite the build env thats some complex bin stubs and magic there i think

it's just a constant in the binary

remember that nixos tools can override this too using some tool which rewrites that constant in some table

ah right