Fork me on GitHub
#tools-deps
<
2019-10-18
>
timovanderkamp10:10:35

Hi, I have a dependency that suddenly broke. I am using https://clojars.org/amazonica and it results in the following error: Unable to resolve com.amazonaws/amazon-kinesis-client version: [1.7.5, 1.8.0). Any ideas what couldve caused this?

markwoodhall10:10:15

We have had the same issue today, it looks like maven-metadata.xml only has one version available now. https://repo1.maven.org/maven2/com/amazonaws/amazon-kinesis-client/maven-metadata.xml, I have asked about it here - https://github.com/awslabs/amazon-kinesis-client/issues/631 but I'm not sure I am asking the right thing. I fixed temporarily by doing:

wget  
wget 
mvn install:install-file -Dfile=amazon-kinesis-client-1.7.5.jar -DgroupId=com.amazonaws -DartifactId=amazon-kinesis-client -Dversion=1.7.5 -Dpackaging=jar -Dpomfile=amazon-kinesis-client-1.7.5.pom.xm

👍 4
Alex Miller (Clojure team)12:10:10

This may really need to be filed with sonatype for maven central

rickmoynihan11:10:14

Is there a way to run clojure -Srepro -Sdeps {,,,} with a flag that prevents it merging the command line deps with a deps.edn file in the working directory?

rickmoynihan11:10:36

i.e. an equivalent of Srepro but for the pwd’s deps.edn

Alex Miller (Clojure team)12:10:23

No, but I am working on a release right now that may help you. What’s the use case?

rickmoynihan13:10:45

Hey… Yes it’s running a clj dev tool we have.

rickmoynihan13:10:10

essentially because it merges with deps.edn we end up running into this tdeps issue: https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-101?filter=allissues&amp;orderby=priority%20DESC&amp;keyword=101 as our deps.edn specifies a :mvn/repos which makes our CI 50s slower than it needs to be

rickmoynihan13:10:48

we pull in the tool over a git dep

rickmoynihan13:10:22

how might the new release help?

Alex Miller (Clojure team)13:10:39

the new feature lets an alias run with isolated rather than merged :deps with the project deps. that's similar but different than what you're wanting

Alex Miller (Clojure team)13:10:54

do you want to completely remove the repo or just tweak it?

Alex Miller (Clojure team)13:10:55

one thing I did happen to add in latest (not yet released) is the ability to remove a repo by setting it's name to nil

Alex Miller (Clojure team)13:10:30

so if that was sufficient you could -Sdeps '{:mvn/repos {"the-server" nil}}'

rickmoynihan14:10:38

completely removing would be fine

rickmoynihan14:10:06

hmm interesting… that might work

rickmoynihan14:10:19

I didn’t think of that

Alex Miller (Clojure team)14:10:03

that will fail on current release, but should work on next release, coming. ... hopefully real soon

rickmoynihan14:10:44

awesome — is real soon likely to be some time today? or next few days?

rickmoynihan14:10:12

no need to promise 🙂

Alex Miller (Clojure team)14:10:19

hopefully today, although it sometimes takes a day for the brew pr to make it through

rickmoynihan14:10:37

yeah stuff usually takes longer than you think

rickmoynihan14:10:28

Anyway, thanks again for this… it’s very timely.

rickmoynihan14:10:28

isolated deps would also be useful too — I’ve wanted that from time to time

rickmoynihan14:10:50

but instead settled for refactoring stuff out of the core deps into aliases — the problem being the default case then suffers with extra alias baggage

Alex Miller (Clojure team)14:10:00

yeah, it is particularly useful for tools

👍 4
zetafish15:10:38

Is there a way to prevent tools-deps to download maven-metadata.xml files?

Alex Miller (Clojure team)15:10:40

are you looking for an "offline" mode?

Alex Miller (Clojure team)15:10:07

if you already have needed versions locally, I'm actually not sure off the top of my head whether it will make any remote calls.

Alex Miller (Clojure team)15:10:05

yeah, I don't think there's any way to do explicitly that right now

zetafish15:10:38

I am using amazonica in deps.edn and explicitly exclusions the kinisis-client-library. It all worked until today that is. The annoying thing is that I do not even want to use the kinesis-client-library.

Alex Miller (Clojure team)16:10:44

exclusions work on the library level, not version level. is it just running into trying to get an older version to do the check?

zetafish16:10:46

Snippet of deps.edn:

com.amazonaws/aws-java-sdk-core    {:mvn/version "1.11.521"}
  com.amazonaws/aws-java-sdk-sts     {:mvn/version "1.11.521"}
  com.amazonaws/aws-java-sdk-s3      {:mvn/version "1.11.521"}
  amazonica                          {:mvn/version "0.3.140"
                                      :exclusions
                                      [com.amazonaws/aws-java-sdk
                                       com.amazonaws/amazon-kinesis-client]}

Alex Miller (Clojure team)16:10:32

yeah, I was more wondering how it manifests in a clj failure?

zetafish16:10:04

% clj -Stree                                                                     
Error building classpath. Unable to resolve com.amazonaws/amazon-kinesis-client version: [1.7.5, 1.8.0)

Alex Miller (Clojure team)16:10:09

I'm guessing amazonica relies on some particular older version and it's trying to find that, before it gets to the point of checking exclusions

zetafish16:10:27

So something thinks that it should find [1.7.5, 1.8.0) and then later it is excluded anyway.

Alex Miller (Clojure team)16:10:12

if you're just looking for a temporary workaround, you may be able to add it at the top level, with the version that's available, and exclude it there too

Alex Miller (Clojure team)16:10:59

com.amazonaws/amazon-kinesis-client {:mvn/version "1.12.0" :exclusions [com.amazonaws/amazon-kinesis-client]}

zetafish16:10:18

That looks really silly but let me try it 🙂

Alex Miller (Clojure team)16:10:12

nah, exclusions works only on children

Alex Miller (Clojure team)16:10:25

clj -Sdeps '{:aliases {:x {:override-deps {com.amazonaws/amazon-kinesis-client {:mvn/version "1.12.0"}}}}}' -A:x

Alex Miller (Clojure team)16:10:42

basically supply an alias that overrides the version to use to the one that happens to be available

Alex Miller (Clojure team)16:10:36

actually, never mind, you'll still hit this

Alex Miller (Clojure team)16:10:44

have you checked any of hte maven mirrors?

zetafish16:10:22

But I guess these mirrors will be updated to central at a certain moment, as they are mirrors.

zetafish16:10:47

Or does each mirror keeps their owne maven-metadata

Alex Miller (Clojure team)16:10:11

oh, not in metadata though

Alex Miller (Clojure team)16:10:14

so you could try adding {:mvn/repos {"central" {:url ""}}}

Alex Miller (Clojure team)16:10:55

that doesn't seem to work for me, but I'm not sure why

zetafish16:10:27

Thanks. That will probably work (until it gets hit by 1.12.0 as well)

Alex Miller (Clojure team)16:10:15

oh, that works if I rm the amazon-kinesis-client out of my ~/.m2 (it caches that source metadata)

Alex Miller (Clojure team)16:10:51

maven central uploads have been giving me many weird errors over the last 24 hrs

Alex Miller (Clojure team)16:10:24

might be something is awry there and an indexing job failed

Alex Miller (Clojure team)16:10:37

did you file a ticket with sonatype?

Alex Miller (Clojure team)16:10:57

I have an acct there already and I can do so if needed

zetafish17:10:04

It would be great if you filed a ticket there.

Alex Miller (Clojure team)13:10:22

looks like it's fixed now

rickmoynihan17:10:11

our CI (travis) has .m2 cached… but it seems that running clojure for the first time on a single private git/dep in each build container refetches these deps each time:

Downloading: org/clojure/clojure/1.10.0/clojure-1.10.0.pom from 
Checking out: [email protected]:Swirrl/omni.git at 7570a53fdbb1d72aabac13ab25e1172ced808eb8
Downloading: hiccup/hiccup/1.0.5/hiccup-1.0.5.pom from 
Downloading: org/clojure/clojure/1.10.0/clojure-1.10.0.jar from 
Downloading: org/apache/commons/commons-lang3/3.5/commons-lang3-3.5.jar from 
Downloading: commons-io/commons-io/2.5/commons-io-2.5.jar from 
Downloading: org/ow2/asm/asm/5.0.3/asm-5.0.3.jar from 
Downloading: hiccup/hiccup/1.0.5/hiccup-1.0.5.jar from 

rickmoynihan17:10:33

I don’t know where they’re coming from… as hiccup is not a dependency of omni; and they already exist in .m2

rickmoynihan17:10:25

so it looks like tools.deps is refetching them

Alex Miller (Clojure team)17:10:09

I can't really do anything without more info. need 1) deps.edn and 2) output of clj -Stree -Sdeps '{:aliases {:v {:verbose true}}}' -A:v

Alex Miller (Clojure team)19:10:29

is this in response to request on @U06HHF230 - are you working together? or are there two things here?

rickmoynihan10:10:38

Not working together no. I tried running with :verbose and it looks like the deps are coming from the git dep after all so I suspect this is another instance of not being able to configure updatePolicy on :mvn/repos, i.e. it looks like clojars is somehow using an updatePolicy of daily rather than never. i.e. I think it’s this issue: https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-101?filter=allissues&amp;orderby=priority%20DESC&amp;keyword=101

rickmoynihan10:10:29

It’d be nice to support this, as it prohibits caching the deps in CI — which slows down the builds

Alex Miller (Clojure team)12:10:37

Daily is the default so that matches what I would expect. Are any of these Snapshots?