Integrant with aero is basically how we solve some of this. We built something that looks like Integrant using component.

Our problem is pretty much that a namespace can blow up if deployed where it can't read certain environment variables and look at certain files. These files change per library, with the same path. I guess there's some convention for the path. This is particularly noticeable in localization.

@rickmoynihan In our team using gradle we define every dependency explicitly, no matter if its an in-team-project dependency or an external one. So naturally I would try the same with leiningen / tools-deps / boot.

@sveri: Not sure exactly what you mean by defining every dependency explicitly? Do you just mean that you produce mvn artifacts for each subproject?

@rickmoynihan gradle has a concept of project dependencies. In the root build file you can define your projects which then can depend on another.

this is kind of a wip / sneak-peek but I’ve implemented a dynamic add-lib that you can use at the repl (that also will download and add transitive deps)

$ clj -Sdeps "{:deps {org.clojure/tools.deps.alpha {:git/url \"" :sha \"d492e97259c013ba401c5238842cd3445839d020\"}}}"
Cloning: 
Checking out:  at d492e97259c013ba401c5238842cd3445839d020
Clojure 1.9.0
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'org.clojure/core.memoize {:mvn/version "0.7.1"})
Downloading: org/clojure/core.memoize/0.7.1/core.memoize-0.7.1.pom from 
Downloading: org/clojure/core.cache/0.7.1/core.cache-0.7.1.pom from 
Downloading: org/clojure/data.priority-map/0.0.7/data.priority-map-0.0.7.pom from 
Downloading: org/clojure/core.memoize/0.7.1/core.memoize-0.7.1.jar from 
Downloading: org/clojure/core.cache/0.7.1/core.cache-0.7.1.jar from 
Downloading: org/clojure/data.priority-map/0.0.7/data.priority-map-0.0.7.jar from 
true
user=> (require 'clojure.core.memoize)
nil
user=> (add-lib 'clj-time {:mvn/version "0.14.3"})
true
user=> (add-lib 'clj-time {:mvn/version "0.14.3"})
false

^^ you can use that command at the top to try it yourself off the branch

everything about this is subject to change as Rich and I are still discussing many aspects of naming and use

there is also a new libmap namespace that tracks the libs currently in use (these can be loaded via a property that points to a file which is set by clj, or by a property set to edn data - hook for tools, or by just inspecting the classpath and trying to understand it)

add-lib will add new libs in the context of the existing libs, so it won’t (for example) download and add libs of transitive deps that have already been loaded. when you add libs, those are added to the runtime lib map and considered in subsequent adds

This is very cool, and useful!

the primary use here is to add a capability to the repl environment to dynamically add libs. this does not give you the isolation of something like pods, but it’s a small useful step into a more live repl experience.

@alexmiller this uses java properties you say?

yes - clojure.libfile (will now be set by clj) and clojure.libmap

although the fallback to classpath introspection actually does a reasonable job even if that’s not set in my experience so far

Pods may conflict with that, as java properties aren't localized to the pod. I don't have an alternative suggestion, but just a data point that occurs to me.

if you’re using pods, you don’t need this

classpath introspection:

I'm thinking of inspection of clojure.libfile for other purposes, e.g. finding gitlibs to ignore during a (reset)

That's a bad example in the context of pods, but I think it conveys my intent.

the clojure.libfile points to the libfile corresponding to the classpath used when starting the jvm. that has a fixed answer.

if you’re using pods, you’re in a bubble under that.

libmap then, rather than libfile?

https://github.com/RutledgePaulV/clj-embed wouldn't be able to convey clojure.libmap to it's runtimes?

libmap is still jvm-wide so like libfile

but I think it’s interesting to consider how to allow classloaders to build their own env

Maybe I don't understand what libmap is, what is it?

it’s just libfile in edn form

so a tool could start the jvm, build the classpath via means other than clj and pass the libmap via data rather than file

Given the definition of "jvm-wide", then java properties are always right. My thoughts are that it's really closer to being about the classloader though, what libmap generated the current classloader.

the runtime libmap is basically stored in a var right now. maybe interesting to consider whether that should be dynamic to allow per-thread context or something per-classloader to handle pod/clj-embed type cases

Sorry, this probably sounds very pedantic. This is really exciting, I'm happy about the feature even as it currently is.

It's definitely notable that due to how Clojure works, embedding always carries a new copy of Clojure. So "clojure local" is perhaps sufficient?

no, I appreciate the good pokes on this. we’ve thought a fair bit about the add-lib stuff but less about the runtime management of the libmap and so I’m still trying to work out the ideal shape of that. and we have other deeper plans for the libmap stuff so I want to get it right.

it’s reasonably likely that part will actually float into Clojure itself

Which part, sorry?

the runtime libmap management (not add-lib)

So clojure.core will be aware of the deps.edn format? :thinking_face:

it’s just a map

so, yes it will be aware of maps :)

in terms of doing semantic things - that doesn’t have to be in core. core’s job is to ensure that it’s loaded and available in a way that can be replied upon

I wouldn’t rule out core being aware of it but I don’t see a need for that yet

well, maybe. we’ll see where things go. :)

maybe I should push this stuff into a blog post

I'm a little confused. Will it be a state storage, which libmap will sit in as a key?

Blog posts are always exciting, even if marked as "these ideas are alpha"

I don’t think I understand the question

right now it’s a var holding an atom holding the libmap

you can look at the code on the branch, it’s not that long

https://github.com/clojure/tools.deps.alpha/compare/add-lib

What will Clojure know about managing the runtime libmap though? I'm really confused by what it means for that to be in Clojure, sorry.

it means that anything using Clojure could look at the libmap (not just things using tools.deps.alpha)

http://insideclojure.org/2018/05/04/add-lib/

The channel name is #tools-deps not #tools.deps BTW @alexmiller (last line of the blog post)

gosh dang it

fixed

it’s a var holding an atom, so would just swap! the atom

but I think there are interesting potential uses that are merely read-only

I understand now. I was interpreting your words too broadly I think. This would solve the problem of pods. If I'm thinking right, this would complicate Clojure becoming aware of class loaders in the future. Admittedly, this is something I am a minority who longs for this.

Clojure only works because of custom classloaders so I don’t think I agree at all with that

I also don’t think this would solve the isolation aspect of pods

I mean that you can't require multiple versions of a namespace from different class loaders. No? The pod could be controlled to update the atom on start couldn't it?

I think you’re misunderstanding what is happening here

but I’m not sure that I understand what you think it’s doing enough to clarify

the libmap is not affecting anything about how the Clojure runtime currently works

it is assisting in opening a path to additively add jars to the DynamicClassLoader in use (this actually has existed in Clojure for a very long time with clojure.core/add-classpath which is deprecated and I think has not worked for a very long time)

pods in Boot provide isolation (different Clojure runtimes) and already do allow you to load different versions of a Clojure namespace at the same time in different pods as far as I understand them

My use of from there was poor. You can't have multiple versions of a library loaded into a single Clojure. You have to load a whole new Clojure.

yes and nothing is changing about that

I’d say that problem is way down the interest list

I know. I was thinking about it that were ever to be "fixed". Yeah, I'm one of the few it bothers because I have interest in a constrained deployment environment, where dependencies aren't really possible.

I totally get why and when isolation is useful, but Clojure was never designed with that intent and there seem to be more important problems to solve

https://twitter.com/mfikes/status/992514389042454529