Hi there, I don't have JIRA access so I can't comment on the issue, but I can report that at Pitch we are still hitting https://clojure.atlassian.net/browse/TDEPS-153 regularly, with version 1.10.3.967 (the version that is supposed to fix it?). Specifically on CircleCI with the runner using this docker image: circleci/clojure:openjdk-8-tools-deps-1.10.3.967-node . We are just now implementing -Sthreads 1 as a workaround, but figure that the bug report might still be helpful

What is the best way to debug why deps aren’t being fetched from private s3 buckets?

I have added my new under :mvn/repos in my deps.edn and have a corresponding ~/.m2/settings.xml

I should add this works for some s3 buckets I already have… I’m just adding another (so I essentially just copy/edited the config of the previous ones)

Also all of the 3 repositories are actually in the same s3 bucket; with the same creds… there are just different top level folders in them partitioning the deps as part of a promotion strategy…

and the original 2 work — but the new 3rd one doesn’t seem to. I should add I deployed the deps via the feature I added last year to deps-deploy… so I may have just misconfigured that somehow; though I can confirm that the dep is in the bucket in what looks to be the right place; but it would be nice to debug it to see which end is at fault (the deployment or the consumption / auth etc.

can you fetch the dep outside the Clojure CLI using aws?

yes

and what happens when you use the cli? error?

clojure  -P
Downloading: com/swirrl/auth0/0.3.build-improvements_local-94a8b78/auth0-0.3.build-improvements_local-94a8b78.pom from swirrl-jars-releases
Downloading: com/swirrl/auth0/0.3.build-improvements_local-94a8b78/auth0-0.3.build-improvements_local-94a8b78.pom from swirrl-jars-branch-builds
Downloading: com/swirrl/auth0/0.3.build-improvements_local-94a8b78/auth0-0.3.build-improvements_local-94a8b78.pom from swirrl-jars-snapshots
Downloading: com/swirrl/auth0/0.3.build-improvements_local-94a8b78/auth0-0.3.build-improvements_local-94a8b78.jar from swirrl-jars-releases
Downloading: com/swirrl/auth0/0.3.build-improvements_local-94a8b78/auth0-0.3.build-improvements_local-94a8b78.jar from swirrl-jars-branch-builds
Downloading: com/swirrl/auth0/0.3.build-improvements_local-94a8b78/auth0-0.3.build-improvements_local-94a8b78.jar from swirrl-jars-snapshots
Error building classpath. Could not find artifact com.swirrl:auth0:jar:0.3.build-improvements_local-94a8b78 in central ()

I’m assuming that is allowed… I’ve done stuff like that in the past

oh actually I pasted a slightly different one there

that was me testing it via a dirty local build (just installed into .m2)

it's certainly not sorted in a way you'd expect for version comparison

true

basically the setup is that branch builds are building immutable artifacts like this

maven version is MAJOR.MINOR.INCREMENT-QUALIFIER - if any of the first 3 is not a number, the whole thing is alpha sorted

ok so I might need an increment too?

and minor and increment are optional

the version has major minor

so just changing the .build to -build would turn the last part into a qualifier

ah gotcha

but I'm not sure that would affect the issue

me neither — but good to know

and it’s something I will fix

in short, I don't know. I would debug it by using tools.deps.alpha and hitting the maven procurer and/or s3 transporter directly

yeah I could certainly do that

is there any internal logging that you’re aware of in the java deps, that I could switch on to get an idea? Oh wait IIRC it now uses the cognitect aws stuff is that right?

don't know, never do that

maven stuff certainly has some logging stuff but I have never bothered to configure it

Ah ok you wrap the cognitect s3 stuff into reified aether classes? https://github.com/clojure/tools.deps.alpha/blob/452b3f215b8f624d17a99ecab22e9e8673e700fc/src/main/clojure/clojure/tools/deps/alpha/util/s3_transporter.clj

that should certainly make it a little easier to debug at a repl

Running this at a REPL:

(clojure.tools.deps.alpha/resolve-deps
   {:deps {'com.swirrl/auth0 {:mvn/version "0.3.0-build-improvements-4302ec2"}}
    :mvn/repos {"swirrl-jars-branch-builds" {:url ""}}} nil)

1. Unhandled clojure.lang.ExceptionInfo
   Could not find artifact
   com.swirrl:auth0:jar:0.3.0-build-improvements-4302ec2 in
   swirrl-jars-branch-builds ()
   {:lib com.swirrl/auth0,
    :coord
    {:mvn/version "0.3.0-build-improvements-4302ec2",
     :deps/manifest :mvn,
     :parents #{[]}}}
                 maven.clj:  167  clojure.tools.deps.alpha.extensions.maven/get-artifact
                 maven.clj:  155  clojure.tools.deps.alpha.extensions.maven/get-artifact
                 maven.clj:  178  clojure.tools.deps.alpha.extensions.maven/eval7338/fn
              MultiFn.java:  244  clojure.lang.MultiFn/invoke
                 alpha.clj:  465  clojure.tools.deps.alpha/download-libs/fn/fn
            concurrent.clj:   35  clojure.tools.deps.alpha.util.concurrent/submit-task/task
                  AFn.java:   18  clojure.lang.AFn/call
           FutureTask.java:  266  java.util.concurrent.FutureTask/run
   ThreadPoolExecutor.java: 1149  java.util.concurrent.ThreadPoolExecutor/runWorker
   ThreadPoolExecutor.java:  624  java.util.concurrent.ThreadPoolExecutor$Worker/run
               Thread.java:  748  java.lang.Thread/run

Looks like I’m running out of time today… I’ll need to pick this up again next week… Many thanks for your sage advice @alexmiller 🙇

you want to close the gap from there to what it's looking for in aws

yes definitely

I can see it’s calling s3-get-object and it’s raising an exception from in there… looks like I mistyped the artifact id

yep that’s it… the real artifact was called com.swirrl/swirrl-auth0

thanks for the help… it’s so much easier to debug the s3 stuff now it’s in clojure! 🙇