Fork me on GitHub
#tools-deps
<
2024-01-28
>
Eugen22:01:24

hi, I think i found a bug / improvement in tools.deps I had a org/eclipse with wrong permissions in ~/.m2/repository/org/eclipse so deps for ring could not be downloaded. However it failed with a not so direct error: I think the error message could be improved to be more specific. It failed with FileNotFound exception when it should fail on trying to download the file. Unfortunatelly I removed eclipse dir before checking permissions. So it might had read permissions but not traverse permissions.

Eugen22:01:49

clojure -Sdeps '{:deps {info.sunng/ring-jetty9-adapter {:mvn/version "0.22.0"}}}'
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See  for further details.
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Warning: failed to load the S3TransporterFactory class
Error building classpath. Failed to read artifact descriptor for org.eclipse.jetty.websocket:websocket-servlet:jar:11.0.15
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.eclipse.jetty.websocket:websocket-servlet:jar:11.0.15
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(Unknown Source)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(Unknown Source)
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(Unknown Source)
        at clojure.tools.deps.extensions.maven$read_descriptor.invokeStatic(maven.clj:115)
        at clojure.tools.deps.extensions.maven$read_descriptor.invoke(maven.clj:106)
        at clojure.tools.deps.extensions.maven$fn__825.invokeStatic(maven.clj:146)
        at clojure.tools.deps.extensions.maven$fn__825.invoke(maven.clj:143)
        at clojure.lang.MultiFn.invoke(MultiFn.java:244)
        at clojure.tools.deps$expand_deps$children_task__463$fn__465$fn__466.invoke(deps.clj:407)
        at clojure.lang.AFn.applyToHelper(AFn.java:152)
        at clojure.lang.AFn.applyTo(AFn.java:144)
        at clojure.core$apply.invokeStatic(core.clj:667)
        at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)
        at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)
        at clojure.lang.RestFn.invoke(RestFn.java:425)
        at clojure.lang.AFn.applyToHelper(AFn.java:156)
        at clojure.lang.RestFn.applyTo(RestFn.java:132)
        at clojure.core$apply.invokeStatic(core.clj:671)
        at clojure.core$bound_fn_STAR_$fn__7194.doInvoke(core.clj:2020)
        at clojure.lang.RestFn.invoke(RestFn.java:397)
        at clojure.lang.AFn.call(AFn.java:18)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.eclipse.jetty.websocket:websocket-servlet:pom:11.0.15 from/to central (): /home/ieugen/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/11.0.15/websocket-servlet-11.0.15.pom.part.lock (No such file or directory)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(Unknown Source)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(Unknown Source)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(Unknown Source)
        ... 25 more
Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.eclipse.jetty.websocket:websocket-servlet:pom:11.0.15 from/to central (): /home/ieugen/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/11.0.15/websocket-servlet-11.0.15.pom.part.lock (No such file or directory)
        at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(Unknown Source)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(Unknown Source)
        at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(Unknown Source)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(Unknown Source)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(Unknown Source)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(Unknown Source)
        ... 28 more
Caused by: java.io.FileNotFoundException: /home/ieugen/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/11.0.15/websocket-servlet-11.0.15.pom.part.lock (No such file or directory)

Alex Miller (Clojure team)15:01:33

I don't understand how to repro this

Eugen15:01:24

hi @U064X3EF3 , IMO this might work:

rm -rf ~/.m2/repository/org/eclipse 
mkdir -p ~/.m2/repository/org/eclipse/
chown root:root ~/.m2/repository/org/eclipse
clojure -Sdeps '{:deps {info.sunng/ring-jetty9-adapter {:mvn/version "0.22.0"}}}'
trying it now

Eugen15:01:09

yes. it's a repro

Eugen15:01:32

IMO the permissions might have changed because of docker conmtainer mounting m2

Eugen15:01:47

ls -la ~/.m2/repository/org/eclipse/
total 8
drwxr-xr-x   2 root   root   4096 ian 29 17:17 .
drwxr-xr-x 149 ieugen ieugen 4096 ian 29 17:17 ..

Alex Miller (Clojure team)15:01:54

you can put this on http://ask.clojure.org if you like and I'll make a jira for it, but I don't think this is a high priority

Alex Miller (Clojure team)15:01:56

since this is all coming out of Maven libs, I'm not sure there's a lot we can actually do about it but I can take a look

Eugen15:01:27

ok, so you think that maven libs are not reporting a failure to write in local maven repo? if they are reporting and tools.deps swallows the exception - it migth be in tools.deps land

Alex Miller (Clojure team)15:01:27

Maven is reporting it - that's what you're seeing (it's just ugly)

Eugen15:01:06

the report mentions file not found. I would expect a "can't write the file" exception first. Unless maven tries to read the lock file before atempting to download and write the deps.

Eugen15:01:12

thanks for the info

Eugen15:01:33

I hope the question will help people in this situation

Alex Miller (Clojure team)15:01:45

this is the first time I've seen this issue come up in the last 5 years

Eugen15:01:17

:man-shrugging:

Eugen15:01:39

is the maven lib not updated in 5y?

Eugen15:01:45

maybe is new behavior

Alex Miller (Clojure team)15:01:55

I don't think anything has changed here. I think something with your Docker config must be either uncommon or other people are sidestepping this problem in some other way. certainly people use Docker with the CLI all the time. (not blaming you, but I do think it's relevant that no one else has ever reported this)

Alex Miller (Clojure team)15:01:10

I'm not a Docker guru, but I know some people use clj -P to download and cache their deps and classpaths before they put them in the container

Eugen15:01:19

sure. just to clarify. I had this issue on host, not inside docker container. I assume the permissions have changed by docker (or me using sudo ?! ) - just assumtions since I don't know

Alex Miller (Clojure team)15:01:25

probably you using sudo

r0man18:01:04

This is a common issue when you mount ~/.m2 and friends into Docker using a volume in order to avoid downloading artifacts all over again. If something gets downloaded inside docker anyhow, that file might be written as root into the users home directory. I also run into this at least a couple of times over the years. Last time last week. 🙂 Not a Docker expert either, and I may be doing something wrong with it. When this happens it is not obvious what happened, and you are luck when you remember that you have been there. :)

Alex Miller (Clojure team)18:01:07

if you want to put that on the http://ask.clojure.org question above, it would help people to find it later (more than here)

r0man18:01:11

I missed there was a question opened. I added it there.

🙏 1