Fork me on GitHub
#babashka
<
2021-07-12
>
hansbugge12:07:52

Is there a way to set the GITLIBS/clojure.gitlibs.dir var within bb.edn such that it is picked up by the clojure command when building the classpath? The idea is to point it to a local directory on CI to allow for dependency caching. (For maven deps it can be handled by adding -Sdeps '{:mvn/local-repo \".local-m2-repo\"} to the command.)

hansbugge12:07:11

Specifying the envvar using the :extra-env param for the clojure command doesn't work, because this affects the environment of the clojure process started with the classpath, not the environment of the process that computes the classpath. Similarly, adding -J-Dclojure.gitlibs.dir=.gitlibs to the command does not work. But surprisingly (to me) it also does not work to set the property directly, as in the following example:

{:tasks
 {foo (do
        (System/setProperty "clojure.gitlibs.dir" ".gitlibs")
        (clojure (str "-Sforce -Sdeps "
                      "'{:deps {io.github.cognitect-labs/test-runner "
                      "         {:git/url \"\""
                      "          :sha \"705ad25bbf0228b1c38d0244a36001c2987d7337\"}}}' "
                      "-Spath")))}}
It does work when setting the envvar for the bb process: GITLIBS=.gitlibs bb foo, but I was hoiping to be able to handle this inside bb.edn.

borkdude12:07:47

Ah, can you make an issue for this?

borkdude12:07:05

I think -J-Dclojure.gitlibs.dir=.gitlibs should at least work. Which version are you on?

borkdude12:07:16

There was a fix regarding option parsing in 0.4.5

hansbugge12:07:22

I'm on 0.4.6. I don't think that will work - it also does not work with the clojure CLI, because the classpath is computed by a different jvm process which does not get java_opts from -J args

borkdude12:07:00

so from the Clojure CLI perspective, what is the expected way of doing this?

hansbugge12:07:40

I don't know! As far as I could tell from inspecting the clojure bash script, there's no way of specifying java_opts for that process

hansbugge12:07:59

so I guess the expected way of doing it is to set GITLIBS in the environment

hansbugge12:07:33

I can make an issue in the babashka project โ€“ feel free to close it if you think it belongs somewhere else

borkdude12:07:39

yes, that's fine

borkdude12:07:15

I would have expected the tools deps step to have picked up that env var

hansbugge12:07:13

Which one? The one from (System/setProperty "clojure.gitlibs.dir" ".gitlibs")?

borkdude12:07:56

the :extra-env one

hansbugge12:07:54

Ah, so (clojure {:extra-env {"GITLIBS" ".gitlibs"}} ,,,)

borkdude12:07:17

yes, that's what I expected to work, but I'm testing it now and it appears it doesn't

borkdude12:07:02

This does work:

{:tasks
 {foo (do
        (shell {:extra-env {"GITLIBS" ".gitlibs"}}
                 (str "clojure -Sforce -Sdeps "
                      "'{:deps {io.github.cognitect-labs/test-runner "
                      "         {:git/url \"\""
                      "          :sha \"705ad25bbf0228b1c38d0244a36001c2987d7337\"}}}' "
                      "-Spath")))}}

borkdude12:07:16

that's invoking the system-installed Clojure version

borkdude12:07:26

but I still would like to have an issue for this so it can be fixed

hansbugge12:07:39

Ah yes, of course. That should be good enough for now, although not as satisfying as using the deps.clj.

hansbugge12:07:59

I'll make an issue.

hansbugge13:07:09

Sorry, I pressed "enter" before I filled out the issue - there's content on it now https://github.com/babashka/babashka/issues/934

borkdude13:07:17

I see the issue, I'm not using babashka.process for the tools deps step

borkdude13:07:26

so the :extra-env option does nothing for that step

hansbugge13:07:07

So the classpath computation is done within the bb process?

hansbugge13:07:33

In that case, shouldn't (System/setProperty "clojure.gitlibs.dir" ".gitlibs") have an effect?

borkdude13:07:43

No, the tools deps step is a separate JVM

borkdude13:07:00

but this JVM isn't started using babashka.process which processes the :extra-env option

borkdude13:07:19

but using some custom ProcessBuilder code that was there since the beginning in deps.clj

borkdude14:07:43

@UNBM99ED7 I think I have a solution. Can I provide you with a binary for testing?

hansbugge15:07:50

Awesome ๐Ÿ™‚ Yes, I'll test

borkdude15:07:22

linux, mac?

hansbugge15:07:59

It works! With (clojure {:extra-env {"GITLIBS" ".gitlib"}} ,,,)

borkdude15:07:37

I'll write a test for it and hopefully this week there will be a new proper release

hansbugge15:07:57

Amazing! ๐Ÿ™‚ Thanks for all your work!

hansbugge15:07:45

That means I can put something like this in my :init job

(def clj
    (if (some-> (System/getenv "CI") (not= "0"))
      (partial clojure
               {:extra-env {"GITLIBS" ".gitlibs"}}
               "-Sdeps '{:mvn/local-repo \".m2-repo\"}'")
      clojure))

borkdude15:07:00

if it's easier for you, you can also provide the deps map as a separate string without the single quotes

๐Ÿ‘ 2
borkdude15:07:06

I think even just the quoted EDN map will work

borkdude15:07:14

since the conversion to string is done automatically

hansbugge15:07:49

Indeed that works. Much prettier

bherrmann14:07:20

I'm confused by this.

$ bb version
babashka v0.4.6
$ cat w.bb 
(+ 3 2)
$ bb w.bb
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  Could not resolve symbol: 
Location: /home/bob/e/devconsole/w.bb:1:1
Phase:    analysis

----- Context ------------------------------------------------------------------
1: (+ 3 2)
   ^--- Could not resolve symbol: 

borkdude14:07:24

hmm, that is surprising indeed. let me try localy

borkdude14:07:54

[email protected] /tmp/foo $ cat /tmp/w.bb
(+ 3 2)
[email protected] /tmp/foo $ bb /tmp/w.bb
5

borkdude14:07:02

Perhaps there is a weird invisible character in your file?

borkdude14:07:59

xxd /tmp/w.bb
00000000: 282b 2033 2032 290a                      (+ 3 2)

bherrmann14:07:42

$ xxd http://w.bb 00000000: efbb bf28 2b20 3320 3229 0a ...(+ 3 2).

grazfather14:07:51

thatโ€™ll do it

bherrmann14:07:09

I see now that emacs thinks the file is "unicode" ...

grazfather14:07:29

which is fine, what encoding? Ascii is a subset of utf-8

grazfather14:07:17

those are byte order marks

bherrmann14:07:25

I wonder how I did that?

borkdude14:07:55

I see something in the background about little endian... which might be related? ;)

bherrmann14:07:26

FYI: To clear the chars I used,

iconv -c -f utf-8 -t ascii input.txt > output.txt