Fork me on GitHub
#babashka
<
2023-03-21
>
István Karaszi21:03:21

I was trying to build the docker image from the master branch but it is failing for me

István Karaszi21:03:24

Execution error (FileNotFoundException) at babashka.impl.common/eval187$loading (common.clj:1).
Could not locate sci/ctx_store__init.class, sci/ctx_store.clj or sci/ctx_store.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

Full report at:
/tmp/clojure-1249190392639535167.edn
Error encountered performing task 'do' with profile(s): 'base,system,user,provided,dev,xml,yaml,csv,transit,httpkit-client,httpkit-server,core-match,hiccup,test-check,selmer,logging,priority-map,reflection'
Suppressed exit
The command '/bin/sh -c ./script/uberjar' returned a non-zero code: 1

István Karaszi21:03:59

I’ve tried with BABASHKA_LEAN=true as well, without much luck

István Karaszi21:03:04

$ docker build --build-arg BABASHKA_LEAN=true -t babashka:slim .

borkdude21:03:47

@UJSBQNUG6 did you clone the babashka repo with --recursive ? It has git submodules

István Karaszi21:03:48

yes, I’ve just realized this in this exact minute 😄

István Karaszi21:03:17

this was the Docker issue I was talking about earlier in another thread

lispyclouds21:03:40

@UJSBQNUG6 thanks for the change, just fyi this reduces the image of the docker image for local consumption and not the ones we push out to the registry. what exactly was the issue you were facing?

István Karaszi21:03:27

well, then I guess we need to modify

István Karaszi21:03:45

or I don’t know which one is pushed to the registry

István Karaszi21:03:51

but IIRC that had the same issue

lispyclouds21:03:13

yeah thats pretty lean anyways. we simply pack the binary in it. the one you changed is where its built

István Karaszi21:03:37

this image has the same issue, it does an apt-get update without removing the package information later

István Karaszi21:03:54

if you check the 7th layer you can see that

lispyclouds21:03:31

ah yes, do you wanna update the PR then? its more useful in http://Dockerfile.ci actually

István Karaszi21:03:11

should I revert my changes in the Dockerfile?

lispyclouds21:03:32

can keep it, if someone is building off it locally

István Karaszi21:03:05

the Alpine image has --no-cache, so I believe that is good

2
István Karaszi22:03:56

although hadolint can see a couple of issues with the images

István Karaszi22:03:32

I can fix those issues if you want

István Karaszi22:03:41

I can see that the /tmp/glibc-2.28-r0.apk is not removed

lispyclouds22:03:01

sure go for it

István Karaszi22:03:48

also - in my opinion - if you are using multi staged builds it is not necessary to combine the RUN commands in the build image

István Karaszi22:03:23

since the size there does not really matter since you are not pushing that anywhere and it actually hurts the build time since if anything changes it cannot use the layer cache

István Karaszi22:03:00

what is the point to install tar in the Alpine image?

borkdude22:03:06

tar is necessary for bb pods

István Karaszi22:03:28

ok, and I guess the Ubuntu image has that already

borkdude22:03:31

if you want the absolute most minimal image, you can start from scratch and just copy the linux static binary

borkdude22:03:45

without changing the bb images at the risk of breaking something

borkdude22:03:45

why is rm graalvm-ce-java19-linux-${GRAALVM_ARCH}-${GRAALVM_VERSION}.tar.gz removed?

István Karaszi22:03:17

because it is not necessary in the build image, as I wrote above

lispyclouds22:03:35

> also - in my opinion - if you are using multi staged builds it is not necessary to combine the RUN commands in the build image you mean the Dockerfile right? agreed but i dont think we use circle's layer cache. is all ephemeral machines

István Karaszi22:03:50

yes, and by local builds

István Karaszi22:03:20

adding rm /tmp/glibc-2.28-r0.apk could further reduce the Alpine image by 2.3MB

👍 2
István Karaszi22:03:21

I won’t do anymore modifications in this PR

István Karaszi22:03:34

we could address the hadolint issues in a separate PR maybe?

lispyclouds22:03:54

yes thats what i was gonna say

István Karaszi22:03:59

some of them are false positives

István Karaszi22:03:03

$ hadolint Dockerfile*
Dockerfile:4 DL3027 warning: Do not use apt as it is meant to be a end-user tool, use apt-get or apt-cache instead
Dockerfile:5 DL3027 warning: Do not use apt as it is meant to be a end-user tool, use apt-get or apt-cache instead
Dockerfile:5 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
Dockerfile:70 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
Dockerfile:71 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
Dockerfile:73 DL3007 warning: Using latest is prone to errors if the image will ever update. Pin the version explicitly to a release tag
Dockerfile:74 DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
Dockerfile:74 DL3015 info: Avoid additional packages by specifying `--no-install-recommends`
Dockerfile.alpine:9 DL3018 warning: Pin versions in apk add. Instead of `apk add <package>` use `apk add <package>=<version>`
Dockerfile.alpine:9 DL3019 info: Use the `--no-cache` switch to avoid the need to use `--update` and remove `/var/cache/apk/*` when done installing packages
Dockerfile.alpine:16 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
Dockerfile.alpine:17 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
Dockerfile.alpine:18 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
Dockerfile.alpine:19 DL3059 info: Multiple consecutive `RUN` instructions. Consider consolidation.
Dockerfile.alpine:24 DL3018 warning: Pin versions in apk add. Instead of `apk add <package>` use `apk add <package>=<version>`
Dockerfile.alpine:24 DL3019 info: Use the `--no-cache` switch to avoid the need to use `--update` and remove `/var/cache/apk/*` when done installing packages
 DL3007 warning: Using latest is prone to errors if the image will ever update. Pin the version explicitly to a release tag
 DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
 DL3015 info: Avoid additional packages by specifying `--no-install-recommends
`

lispyclouds22:03:29

yeah these are too pedantic ones

István Karaszi22:03:55

the first too makes sense

lispyclouds22:03:29

lets do it all in a different PR, this one is in a good bounded context

👍 2
lispyclouds22:03:40

looks fine to me, waiting for the ci run to finish

👍 2
lispyclouds22:03:56

also could you add a changelog entry too? can make a unreleased section in https://github.com/babashka/babashka/blob/master/CHANGELOG.md

lispyclouds22:03:00

now that i look at it why is WORKDIR "/usr/local/bin" needed?

lispyclouds22:03:37

all the paths are absolute there

lispyclouds22:03:32

that effectively changes the pwd of the bb CMD right?

István Karaszi22:03:57

WORKDIR will create the directory for us

István Karaszi22:03:01

true, I was not thinking about that

borkdude22:03:11

please don't make any changes that aren't necessary to the problem statement that goes with this PR

lispyclouds22:03:41

yeah thats not needed

lispyclouds22:03:23

lets stick to mkdir, the other one is effectively a breaking change in some sense

István Karaszi22:03:46

true, I am sorry

lispyclouds22:03:39

no worries! good that we can catch these things early on 😄

👍 2
borkdude22:03:49

ok, LGTM too, thanks @UJSBQNUG6!

🙏 2
lispyclouds22:03:59

i'll merge it?

borkdude22:03:03

I don't think the docker step is ran in branches right? so waiting for CI might not make sense

István Karaszi22:03:31

thank you guys for the super quick merge

russmatney23:03:04

I'm seeing this when running babashka after upgrading to the latest, but only when bb.edn has :deps to install (it works fine outside of my repo):

❯ bb -e '(print "hi")'
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main
Exception in thread "main" java.io.FileNotFoundException:
/home/russ/.clojure/.cpcache/954D5CED186DE6EB3C969EE7C387062C.cp (No such file or directory)
It seems to fail anytime it needs to use java to install deps. (commenting out the deps in my bb.edn allows the above command to work again). I've hit this a few times before, and usually i delete a few .cpcaches (in the project and in ~/) and things start working again, but it's been a while, and I feel like I'm forgetting a step in this process... are there other caches I need to delete or other workarounds for this?

borkdude23:03:40

you should not have to delete any caches when upgrading, but if you want to by pass the caches, you can pass bb -Sforce

borkdude23:03:06

if you can make a repro of this, if possible, that would be helpful

russmatney23:03:47

sure - i think i messed up my bb/clojure/java versions somehow, trying to get to the bottom of it now

borkdude23:03:02

what does bb --version say?

russmatney23:03:25

❯ bb --version                                                                                                                                                                       7:10PM
egrep: warning: egrep is obsolescent; using grep -E
babashka v1.3.176

russmatney23:03:55

and clojure looks like:

~/russmatney
❯ bb clojure --version                                                                                                                                                               7:11PM
egrep: warning: egrep is obsolescent; using grep -E
Clojure CLI version (deps.clj) 1.11.1.1257

~/russmatney
❯ clojure --version                                                                                                                                                                  7:11PM
egrep: warning: egrep is obsolescent; using grep -E
Clojure CLI version 1.11.1.1224

russmatney23:03:00

i'm wondering if those versions should align

russmatney23:03:10

1224 vs 1257

borkdude23:03:20

no need to align

👍 2
borkdude23:03:18

if you have deleted your .m2 or .gitlibs folder then it could be that you would get such an exception, but -Sforce should fix that

russmatney23:03:38

gotcha - i did delete .m2 at some point, but forgot .gitlibs

borkdude23:03:50

and you can ditch home .clojure/.cpcache + local-dir .cpcache as well, in doubt

russmatney23:03:24

bah, still the same error with all those cleared. must be something else that's wrong 😕

russmatney23:03:50

thanks for the help anyway - i'll dig around a bit

borkdude23:03:08

if you put the same deps from bb.edn in a deps.edn and start clj, does it work?

russmatney23:03:25

yeah - the deps install and i hit a repl

russmatney23:03:35

then moving things back and running bb again, same crash

borkdude23:03:20

and with bb -Sforce?

russmatney23:03:50

bb -Sforce hits the same error

russmatney23:03:53

bb -Sforce                                                                                                          7:22PM
egrep: warning: egrep is obsolescent; using grep -E
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main
Exception in thread "main" java.io.FileNotFoundException: /home/russ/.clojure/.cpcache/1621164B24C75150EF237DB929581D8A.cp (No such file or directory)
  

borkdude23:03:06

what is that egrep thing

russmatney23:03:10

testing in babashka/fs now to be sure it's not the repo

russmatney23:03:14

the egrep is noise, sorry

russmatney23:03:26

a zsh error i haven't gotten around to yet

borkdude23:03:49

what is in ~/.deps.clj ?

russmatney23:03:16

❯ ~/.deps.clj/                                                                                                        7:22PM
./            1.10.3.1040/  1.11.1.1149/  1.11.1.1165/  1.11.1.1200/  1.11.1.1237/
../           1.11.1.1113/  1.11.1.1155/  1.11.1.1182/  1.11.1.1208/  1.11.1.1257/

borkdude23:03:35

and in 1.11.1.1257?

russmatney23:03:56

❯ l ~/.deps.clj/1.11.1.1257/ClojureTools/                                                                             7:24PM
./                             clojure-tools-1.11.1.1257.jar  tools.edn
../                            example-deps.edn               tools.zip

russmatney23:03:03

just the ClojureTools dir

borkdude23:03:13

maybe throw away that dir, long shot

russmatney23:03:40

well hey!

🏆 2
russmatney23:03:43

i think that did it

russmatney23:03:07

instead bb -Sforce showed

Clojure tools not yet in expected location: /home/russ/.deps.clj/1.11.1.1257/ClojureTools/clojure-tools-1.11.1.1257.jar
Downloading  to /home/russ/.deps.clj/1.11.1.1257/ClojureTools/tools.zip
Unzipping /home/russ/.deps.clj/1.11.1.1257/ClojureTools/tools.zip ...
Successfully installed clojure tools!

borkdude23:03:08

maybe the download of the tools jar didn't work correctly the first time

russmatney23:03:46

confirming it's working in the original repo! yay!

russmatney23:03:56

thanks so much @U04V15CAJ 😄

🎉 2
russmatney23:03:25

it's slightly devastating when bb fails on me b/c i lose all my key bindings and window-management powers, haha

russmatney23:03:31

but i still love it!

borkdude23:03:02

maybe I should add a sha256 check or so for that tools jar download

borkdude23:03:37

anyway, glad it's solved

sheepy 2