This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-01-12
Channels
- # announcements (2)
- # babashka (26)
- # beginners (48)
- # calva (32)
- # cider (23)
- # clj-kondo (61)
- # cljfx (3)
- # clojure (93)
- # clojure-australia (2)
- # clojure-europe (23)
- # clojure-losangeles (1)
- # clojure-nl (5)
- # clojure-uk (4)
- # clojurescript (46)
- # cloverage (9)
- # code-reviews (1)
- # copenhagen-clojurians (1)
- # cursive (39)
- # data-science (6)
- # datahike (8)
- # deps-new (8)
- # depstar (2)
- # etaoin (1)
- # fulcro (2)
- # funcool (2)
- # graalvm (5)
- # jackdaw (3)
- # java (17)
- # jobs-discuss (43)
- # kaocha (2)
- # leiningen (25)
- # malli (8)
- # minecraft (1)
- # missionary (8)
- # observability (6)
- # off-topic (37)
- # other-languages (12)
- # practicalli (1)
- # reagent (4)
- # releases (78)
- # remote-jobs (1)
- # sci (9)
- # shadow-cljs (13)
- # spacemacs (6)
- # sql (1)
- # tools-deps (30)
- # xtdb (3)
Can a tool dependency easily access its own version data? As git deps appear to need a SHA, I can’t hardcode the version.
For context, I’m building a tool that bootstraps a bb.edn file, which itself needs a dependency.
access from where?
as in, you are running code from the tool and want to know what version of itself that is?
Yep, that’s right. It’s so I can tell Babashka via bb.edn that this is the version of the library it should reference.
there is not a public api (yet!) but this hack will work:
(-> (System/getProperty "clojure.basis") slurp clojure.edn/read-string :libs (get 'org.clojure/clojure) :mvn/version)
Thanks! That appears to work!
or at the end, check :git/tag
, :git/sha
etc
Not sure if this is "known issue" or not: in CI (and pretty much only in CI), we are seeing intermittent "Failed to read artifact" errors from the aether stuff in the CLI (stacktrace in thread). Is this the multi-threading/race condition issue?
Error building classpath. Failed to read artifact descriptor for ring:ring:jar:1.9.4
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for ring:ring:jar:1.9.4
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:259)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:175)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
at clojure.tools.deps.alpha.extensions.maven$read_descriptor.invokeStatic(maven.clj:115)
at clojure.tools.deps.alpha.extensions.maven$fn__1127.invokeStatic(maven.clj:143)
at clojure.tools.deps.alpha.extensions.maven$fn__1127.invoke(maven.clj:143)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at clojure.tools.deps.alpha$expand_deps$children_task__770$fn__772$fn__773.invoke(alpha.clj:406)
at clojure.tools.deps.alpha.util.concurrent$submit_task$task__479.invoke(concurrent.clj:35)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact ring:ring:pom:1.9.4 from/to clojars ( ): /root/.m2/repository/ring/ring/1.9.4/ring-1.9.4.pom.part (No such file or directory)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:425)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:229)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:207)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:244)
... 13 more
Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact ring:ring:pom:1.9.4 from/to clojars ( ): /root/.m2/repository/ring/ring/1.9.4/ring-1.9.4.pom.part (No such file or directory)
at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:52)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:369)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:628)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:262)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:514)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:402)
... 16 more
Caused by: java.io.FileNotFoundException: /root/.m2/repository/ring/ring/1.9.4/ring-1.9.4.pom.part (No such file or directory)
at java.base/java.io.FileInputStream.open0(Native Method)
at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
at org.eclipse.aether.internal.impl.DefaultFileProcessor.copy(DefaultFileProcessor.java:163)
at org.eclipse.aether.internal.impl.DefaultFileProcessor.copy(DefaultFileProcessor.java:151)
at org.eclipse.aether.internal.impl.DefaultFileProcessor.move(DefaultFileProcessor.java:252)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:482)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:364)
... 21 more
Re-running the failed step nearly always succeeds (and the "failed" artifact is random from build to build).
Mostly, yes, but it drifts as we update deps, and then BitBucket deletes and recreates the cache about every two weeks (to make it fresh again).
Don't think that matches anything I've seen
ring stuff will be coming from clojars right?
It looks like that is in the maven stuff that downloads then copies into place and it is missing the download. Is it possible bitbucket deletes out from under you?
No. This all happens in a single step. I don't know that it is only Clojars deps -- I think I've seen it from Maven deps too -- but I'll keep a closer eye on that in future.
Good to know this isn't a known-but-supposedly-fixed issue 🙂
there are general issues with Maven concurrent downloads that they've been working on for Maven 4
is there any chance of concurrent processes hitting this same repo or it for sure just 1?
I'm not using -Sthreads 1
but otherwise it is just this one process.
do we know how bitbucket shares the m2 cache? if all the builds should have access to m2, what happens when multiple builds are running at once?
(found the docs, directory is just and restored after a successful build, no shared filesystem or anything like that)
Was this problem solved in recent versions? I’m seeing the exact same issue running on org.clojure/tools.deps.alpha "0.14.1178"
Also seeing this discussion in https://clojurians.slack.com/archives/C6QH853H8/p1646073532171879?thread_ts=1646073499.713759&cid=C6QH853H8.
What version is your CLI? clojure -version
Looks like you're using https://clojure.org/releases/tools#v1.11.1.1113 which was April 2022. The latest is 1.11.1413, which uses tools.deps
0.18.1354 @UHS6PHL31
Thanks! I will try bumping the version just wanted to make sure the problem is fixed there.
And yes, you’re correct, we’re using that version of clojure CLI.
We're on the latest CLI (1058).