Clojurians Log v2

you may want to account for the build files themselves changing, not just src

mistake I made in shadow-cljs caching code. only accounting for the actual sources, not the code or config that compiled it 😛

yeah

also local/roots transitive changes

minor enhancement ;)

I have a very strange issue with b/compile-clj. Some builds just work fine, but randomly sometimes I see:

Syntax error (IllegalAccessError) compiling at (clojure/tools/reader.clj:1:1).
read-char does not exist

you can follow some commits over here: https://github.com/clj-easy/graal-build-time/commits/debug I'm adding empty commits named debug just to trigger the build and keep the logs available (when you re-trigger a build the logs get lost)

when the compile fails, it will leave the entrails in the tmp directory if you'd like to examine them

not sure if you have access to that

it seems it doesn't even enter the compile-clj function, I included the source and added some printlns.

(defn compile-clj
  [{:keys [basis src-dirs compile-opts ns-compile filter-nses class-dir sort] :as params
    :or {sort :topo}}]
  (prn :params params)

when the build does succeed, I do see those printlns

(defn compile-sources [_]
  (println "Compiling sources")
  (if (bs/needs-compile?)
    (do
      (prn "basis:")
      (prn (keys (:classpath basis)))
      (println "Compiling Clojure sources.")
      (b/compile-clj {:basis basis
                      :src-dirs bs/sources
                      :class-dir class-dir
                      :ns-compile '[clj-easy.graal-build-time]})
      (println "Done compiling Clojure sources.")
      (println "Compiling java sources.")
      (b/javac {:src-dirs bs/sources
                :class-dir class-dir
                :basis with-svm-basis
                :javac-opts ["-source" "8" "-target" "8"]})
      (println "Done compiling java sources."))
    (println "All up to date, nothing to compile.")))

Syntax error (IllegalAccessError) compiling at (clojure/tools/reader.clj:1:1).
read-char does not exist

can you run the cli with -J-Dclojure.main.report=stderr ?

yeah

Seems very similar to what I hit in this comment thread: https://ask.clojure.org/index.php/10905/control-transient-deps-that-compiled-assembled-into-uberjar?show=10908#c10908

that's a protocol function, maybe an ordering issue during compilation?

some strange stuff happening in this build: https://github.com/clj-easy/graal-build-time/runs/3576160312

I noticed you also had multiple create-basis, which was triggering a race condition on an older version of tools.deps that I had on my machine.

exactly!

I'm doing multiple bases

on latest of tools.build / tools.deps or older?

latest

I'll double check

actually, is tools.build using latest? don't know

good question

Interesting. I don't see this issue on the latest tools.deps, but definitely did on an older one.

no, it's using older

I see...

org.clojure/tools.deps.alpha {:mvn/version "0.12.1036"}

is latest, you can add that to your build alias

ok

thanks!

(I thought I was going mental)

If you can't upgrade for some reason, hiredman's solution in the thread I linked should work. Just preload the troublesome namespaces at the top of the build.clj.

cool, yeah, upgrading is possible

I don't think that would do anything - compilation is in a forked jvm

oh, sorry basis creation

yeah

in the prior maven core version, they removed all the synchronization stuff, we've been having a lot of problems since then, no surprise

the latest tools.deps moves to the fixes for that, and also cleans up a bunch of things where tools.deps was redoing work (some accidentally, some knowingly)

released new tools.build v0.3.0 e418fc9 with the dep update

thanks!

should make builds faster too!

@alexmiller the issue happened again: https://github.com/clj-easy/graal-build-time/runs/3576314900

this time it seems to happen while finding namespaces

perhaps it triggers compilation because of some top level compile that gets evaluated?

I'll try :ns-compile

didn't help.

why does it even invoke find-ns when I provide :ns-compile, this seems weird

managed to trigger another one (different trace): https://github.com/clj-easy/graal-build-time/runs/3576363220

that has a similar error, but doesn't fail the build

Yes

That is the race mentioned earlier

There is another thread that maven kicks off, and it calls back into clojure

Which causes a race on code loading

The reason you got a stacktrace and it didn't fail the build is it happened on the other thread that time

yeah, I figured that. hrm... I could just script the compilation manually for now

I only need a classpath basically to do that

and javac

https://ask.clojure.org/index.php/10905/control-transient-deps-that-compiled-assembled-into-uberjar?show=10913#c10913 as I suggest here you can also preload the code (by requiring the right namespace) to avoid the race

ok, I'll try

shouldn't this s3 / aws stuff be split out of tools.deps.alpha perhaps? I mean, the majority of projects probably don't even use this

we use it at work, but I also know many projects that don't

It is tricky, because it needs to be on the classpath for tools.deps to use if it is going to use it, but tools.deps is the tool for building classpaths

true

but wait a minute, that's true for the tools jar used by the CLI, but not for tools.build, as tools.build is just a library which you can extend by adding more libraries into the alias

Ah, of course right, and tools.deps is just another library being used as part of the build there

we consider the s3 support to be a core feature, so not going to split it out

but clearly it's time to circle back to this parallel load stuff and reeexamine