I hit the Maven 429 Too Many Requests error again today. Obvs I can likely get around that with -Sthreads 1 but I thought this was addressed in tools.deps now, with auto-back-off and retries? Stacktrace in 🧵
Error building classpath. Failed to read artifact descriptor for org.clojure:clojure:jar:1.13.0-alpha10
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.clojure:clojure:jar:1.13.0-alpha10
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:254)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:183)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:269)
at clojure.tools.deps.extensions.maven$read_descriptor.invokeStatic(maven.clj:117)
at clojure.tools.deps.extensions.maven$fn__1297.invokeStatic(maven.clj:145)
at clojure.tools.deps.extensions.maven$fn__1297.invoke(maven.clj:145)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at clojure.tools.deps$expand_deps$children_task__976$fn__978$fn__979.invoke(deps.clj:303)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1994)
at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1994)
at clojure.lang.RestFn.invoke(RestFn.java:428)
at clojure.lang.AFn.applyToHelper(AFn.java:156)
at clojure.lang.RestFn.applyTo(RestFn.java:135)
at clojure.core$apply.invokeStatic(core.clj:671)
at clojure.core$bound_fn_STAR_$fn__5837.doInvoke(core.clj:2024)
at clojure.lang.RestFn.invoke(RestFn.java:400)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:330)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at java.base/java.lang.Thread.run(Thread.java:1527)
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: The following artifacts could not be resolved: org.clojure:clojure:pom:1.13.0-alpha10 (absent): Could not transfer artifact org.clojure:clojure:pom:1.13.0-alpha10 from/to central ( ): status code: 429, reason phrase: Too Many Requests (429)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:474)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:261)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:243)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:243)
... 23 more
Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.clojure:clojure:pom:1.13.0-alpha10 from/to central ( ): status code: 429, reason phrase: Too Many Requests (429)
at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:44)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:419)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:260)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:550)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:450)
... 26 more
Caused by: org.apache.http.client.HttpResponseException: status code: 429, reason phrase: Too Many Requests (429)
at org.eclipse.aether.transport.http.HttpTransporter.handleStatus(HttpTransporter.java:662)
at org.eclipse.aether.transport.http.HttpTransporter.execute(HttpTransporter.java:532)
at org.eclipse.aether.transport.http.HttpTransporter.implGet(HttpTransporter.java:479)
at org.eclipse.aether.spi.connector.transport.AbstractTransporter.get(AbstractTransporter.java:64)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:484)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:416)
... 29 moreFWIW, I just re-ran the BitBucket job and it downloaded everything just fine...
what version of the CLI?
Hey, it's me! Always the very latest stable version 🙂
Clojure CLI version 1.12.6.1673
I only hit this occasionally (not even every time BitBucket rebuilds the cache, which would be once a week). And re-running the same BitBucket job—which would rebuild the cache from scratch again in this case—always seems to work, so it's very infrequent.
what Java version?
as far as I understand it, the current version of CLI which uses the newer version of the Maven resolver which uses the Java HTTP client (Java 11+), which responds appropriately to 429 requests
Again, the very latest Java/JVM version: 27 (27+35 right now but it will be 27.0.1 soon).
the retries in the case of a 429 are not infinite of course - I think the default config is something like retry 3 times on a 5 second interval
so you may still be breaching the retry count even with the latest code. there are properties that can be set to change those. they are per-repo but I'm not exactly sure where you would set them (maybe settings.xml, maybe system props)
these are the properties you're trying to reach https://github.com/apache/maven-resolver/blob/master/maven-resolver-api/src/main/java/org/eclipse/aether/ConfigurationProperties.java#L381-L407
something like -Daether.connector.http.retryHandler.count.central=10 maybe (same for clojars) ?
to set those on a Clojure CLI command line, you want that to apply to the deps process (not the main), so in env on export CLJ_JVM_OPTS="-Daether.connector.http.retryHandler.count.central=10 -Daether.connector.http.retryHandler.count.clojars=10"
and maybe you need to increase the interval with ...retryHandler.interval=10000
aether.transport.http.retryHandler.count.central I think. Thanks. If it happens again, I may try adding that.