This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-03-25
Channels
- # announcements (8)
- # aws (50)
- # aws-lambda (6)
- # babashka (25)
- # beginners (119)
- # bristol-clojurians (5)
- # calva (25)
- # chlorine-clover (23)
- # cider (6)
- # cljs-dev (125)
- # clojure (63)
- # clojure-austin (1)
- # clojure-belgium (1)
- # clojure-dev (48)
- # clojure-europe (11)
- # clojure-italy (2)
- # clojure-nl (5)
- # clojure-spec (3)
- # clojure-uk (66)
- # clojurescript (14)
- # core-logic (5)
- # datomic (13)
- # emacs (10)
- # events (2)
- # fulcro (37)
- # graalvm (11)
- # hoplon (95)
- # jobs-discuss (9)
- # juxt (11)
- # kaocha (16)
- # meander (13)
- # off-topic (24)
- # pedestal (4)
- # re-frame (36)
- # reagent (10)
- # reitit (15)
- # ring-swagger (5)
- # shadow-cljs (23)
- # spacemacs (2)
- # sql (13)
- # tools-deps (32)
- # xtdb (11)
tools.deps is fundamentally about making classpaths, and incidentally about transitive dependency resolution
clj is about running programs
I’ve been meaning to speak about this issue for a while; not sure if others have run into it or not. I have a private library in a private github repo; that we’re requiring via tools.deps. Every so often, in circumstances I don’t fully understand, but it usually appears to be when I do things like change branches (and I suspect the sha in that dep has changed) I get the following exception:
When it occurs I usually spot that my ssh-agent is somehow no longer running; so restart that but then get:
Then clearing the lock file fixes it
We’ve had people report this but I’ve never been able to repro it
It happens to me quite often
there are a few other things that might be relevant:
1. some-private-lib
is actually included in our deps.edn
multiple times; with different :deps/root
s
2. In dev I start a bunch of services with overmind a Procfile process manager. It boots them in parallel. One of those services is my clj
repl; another is shadow-cljs
that references the same deps.edn
; so when resolving deps two processes are occaisionally concurrently resolving/fetching deps in parallel.
I suspect 2 might be problematic… so sometimes try and resolve the deps first; then run overmind… though I do at times forget — it usually seems to work without any apparent problems, though it does seem a bit dangerous on my part.
I saw the first error just the other day. I had two small projects, one using a github ref using one hash (that checked out the repo fine) and then moved to another with a newer hash (but the same repo) and saw the above error when trying to build. I went into the directory and deleted the repo and tried again and it worked. Is that supposed to work?
I think I’ve seen it a similar scenario to that too
but this one is definitely within a single project — though changing branches; with a changing sha on the dep which probably amounts to the same thing
with the newer clj's, resolution happens in parallel, so even in a single project, you could encounter that
you can turn that off with -Sthreads 1
Ok but this is in a single project; but with two concurrent processes potentially resolving the same deps with tools.deps. Is that safe if the processes aren’t coordinated?
don't know
The first error that is
Hi, I love the ability of tools-deps to target git repos at a certain hash. I've operated under the assumption that the hostname of the repo will stay the same forever for private repo dependencies. But that's proven not always to be the case. If a repo moves to a new hostname, it breaks every historical project state that referenced it. Yocto has a feature called a "repo manifest" that allows the repository configuration to be a lot more fluid, and effectively solves this problem. Does tools-deps have an equivalent option?
what is the repo manifest?
got a link or something?
I think this is what I'm talking about: https://www.yoctoproject.org/docs/2.7/bitbake-user-manual/bitbake-user-manual.html#repo-fetcher And an example: https://github.com/texierp/yocto-bsp-manifest/blob/master/default.xml
I suppose there are workarounds for the issue at the OS-level, but this would be a pretty slick feature.
@alex.joseph.whitt is this like having a proxy or alternative source for a repo? I'm struggling to understand.
I'm not a Yocto user myself, but I believe the idea is that you can have a locally-defined manifest of repo sources. For each dependency, Yocto will look through them all in order and try to resolve the dependency in each. That solves the problem of trying to build an old version of your project after your git server hostname changes. The problem is really only sticky when you have multiple internal Clojure dependencies that all point to the same hostname. You have to go through each dependency, branch off of the contemporary commit, update the hostname, commit, then update the hashes on projects that pull in that dependency. Kind of hellish unless you redirect the old hostname at the DNS level or something.
I mean, you are defining a lib and a coordinate (with git config) in your deps.edn. You can change that coordinate/git config but keep the lib name to point it elsewhere.
I guess there could be a layer of indirection added in front of the git url
needs a lot more thought