2021-09-16 tools-deps | Clojure Slack Archive

tools-deps

2021-09-16T19:29:38.268700Z

Running by repl, I get repeatedly the same error message about a certain jar:

Error building classpath. Could not acquire write lock for 'artifact:org.bytedeco:mkl:2020.3-1.5.4'
java.lang.IllegalStateException: Could not acquire write lock for 'artifact:org.bytedeco:mkl:2020.3-1.5.4'

borkdude 2021-09-16T19:33:48.268900Z

concurrency issue? -Sthreads 1?

2021-09-16T19:47:09.269700Z

Indeed, this seems to fix it. I remmber that the jars are big, so individaul download takes long. rw-r--r-- 1 carsten carsten 219M Sep 3 21:35 mkl-2020.3-1.5.4-linux-x86_64-redist.jar -rw-r--r-- 1 carsten carsten 172M Sep 3 21:28 mkl-2020.3-1.5.4-windows-x86_64-redist.jar -rw-r--r-- 1 carsten carsten 147M Sep 3 21:34 mkl-2020.3-1.5.4-macosx-x86_64-redist.jar -rw-r--r-- 1 carsten carsten 137M Sep 3 21:34 mkl-2020.3-1.5.4-linux-x86-redist.jar -rw-r--r-- 1 carsten carsten 112M Sep 3 21:33 mkl-2020.3-1.5.4-windows-x86-redist.jar -rw-r--r-- 1 carsten carsten 4.4M Sep 3 21:25 mkl-2020.3-1.5.4-linux-x86_64.jar -rw-r--r-- 1 carsten carsten 4.1M Sep 3 21:28 mkl-2020.3-1.5.4-linux-x86.jar -rw-r--r-- 1 carsten carsten 4.1M Sep 3 21:25 mkl-2020.3-1.5.4-macosx-x86_64.jar -rw-r--r-- 1 carsten carsten 3.3M Sep 3 21:25 mkl-2020.3-1.5.4-windows-x86_64.jar -rw-r--r-- 1 carsten carsten 2.0M Sep 3 21:28 mkl-2020.3-1.5.4-windows-x86.jar

2021-09-16T19:47:39.270300Z

So it seems that clj has a reproducible issue with those.

2021-09-16T19:48:04.270800Z

I have it inside a Docker, so on an empty maven folder.

2021-09-16T19:48:18.271100Z

and it happens on every run

2021-09-16T19:48:44.271600Z

(or docker build, better said)

Alex Miller (Clojure team) 2021-09-16T19:53:40.271800Z

I'm going to be looking at some of this stuff tomorrow

2021-09-16T20:08:15.272600Z

I have it on every run against an empty .m2 folder with this deps.edn

{
 :mvn/repos {"clojars" {:url ""}
             "central" {:url ""}
             "bedatadriven" {:url ""}}

 :paths ["src" "resources"]

 :deps {
        org.clojure/clojure      {:mvn/version "1.10.3"}
        clj-python/libpython-clj {:mvn/version "2.000"}
        scicloj/clojisr          {:mvn/version "1.0.0-BETA19"}
        scicloj/tablecloth       {:mvn/version "6.012"}
        scicloj/notespace        {:mvn/version "3-beta9"}
        scicloj/scicloj.ml       {:mvn/version "0.1.0-beta4"}}}

Alex Miller (Clojure team) 2021-09-17T21:52:41.276300Z

Hey Carsten, I tried a lot of stuff today and I was able to reproduce this exactly once. The message you were seeing comes out of Maven itself. In a recent lib update I moved the maven resolver libs up to 1.7.x but those versions are not necessarily guaranteed to work with Maven core 3.8.x (they are expecting Maven core 4.0 alphas). Resolver 1.7 has been working on replacing the locking/synchronization infrastructure which is directly in this area. Anyhow, I have fallen back to 1.6.x which should work with the core libs I'm using. If you wanted to try with Clojure CLI 1.10.3.981, I'd be interested to know if that helped or not.

2021-09-20T16:10:06.286200Z

This Dockerfile fails for me everytime : https://gist.github.com/behrica/723cb69c5524497599fb90f7942f153b

Alex Miller (Clojure team) 2021-09-20T16:11:39.286400Z

thanks, that will help

Alex Miller (Clojure team) 2021-09-20T16:12:18.286600Z

one thing that I think contributes here is a lot of the deps under http://scicloj.ml are really big native libs

Alex Miller (Clojure team) 2021-09-20T16:12:30.286800Z

I suspect the size == longer download time which exacerbates the problem

2021-09-20T16:12:46.287Z

Iyes, the MKL libs are very big

2021-09-20T16:12:58.287200Z

> several jars of > 100 MB

2021-09-20T16:13:24.287400Z

error building classpath. Could not acquire write lock for 'artifact:org.bytedeco:mkl:2020.3-1.5.4' java.lang.IllegalStateException: Could not acquire write lock for 'artifact:org.bytedeco:mkl:2020.3-1.5.4'

Alex Miller (Clojure team) 2021-09-20T16:13:52.287600Z

yeah, that exception is in Maven

2021-09-20T16:13:57.287800Z

It works with this clj -P -Sthreads 1

2021-09-20T16:14:06.288Z

always

2021-09-20T16:14:26.288200Z

i will quickly try with 1.10.3.981,

Alex Miller (Clojure team) 2021-09-20T16:14:26.288400Z

they have completely rewritten this area in the newest Maven resolver, but it does not work with the current released version of Maven core

Alex Miller (Clojure team) 2021-09-20T16:15:41.288600Z

in the latest I've fallen back to prior Maven resolver version

2021-09-20T16:15:42.288800Z

docker build running, it takes some minutes

2021-09-20T16:35:21.289200Z

It works with 1.10.3.981

Alex Miller (Clojure team) 2021-09-20T16:38:00.289400Z

oh really

Alex Miller (Clojure team) 2021-09-20T16:38:33.289600Z

well that took their new code out of the loop, so that's good to know

Clojurians Log v2

tools-deps