The first scenario that comes to my mind is a monorepo but Alex has asked for the problem statement to be couched in more general terms.

is this related to the CLJ_CONFIG + mvn-pom problem?

So I'd state the problem in these terms: 1. a development team works together on multiple projects that each have their deps.edn file, and 2. those developers have their own preferred tooling configured in their own ~/.clojure/deps.edn file, and 3. the team as a whole wants a level of consistency in terms of certain library or tooling versions across multiple projects, by providing some standardized aliases and some "pinned" versions (e.g., :override-deps). Currently, 1. and 2. are supported by the CLI/t.d.a. but there's nowhere to hang the multi-project standardization.

@borkdude no, not really. that problem is about how to convey cp modifications to -X programs

Currently, folks who want 3. are using CLJ_CONFIG to provide a team-standard deps.edn file but losing 1. in the process.

The cp/deps modification to -X programs is a separate (but also a real) issue that would benefit from some sort of standardized approach that could be used by tooling in general (not just mvn-pom). So let's keep that discussion separate.

@borkdude Does my three-point problem statement match up with your CLJ_CONFIG use case, or is yours a separate scenario?

1) yes, 2) yes. 3) we use :default-deps for this with nil versions (as documented).

Right now it works well enough for me. Any tools I need for development I can also just put in our work repo's deps.edn. But it would be nice if I should not have to do that.

In clj-kondo I have a concept called config-paths which is a list of dirs that possibly have a configuration for clj-kondo to be merged in. It's similar to CLJ_CONFIG, but CLJ_CONFIG only supports one and not a list of dirs.

Basically a classpath for config.

And your home dir's clj-kondo config is implicitly part of your config paths, unless you opt out of that

I put a few constraints to get multiple unified deps configs to work well, i.e. subrepo aliases must be namespaced, since that makes them simple to merge and call from another directory

@UPH6EL9DH Interesting, but it looks/feels clunky as a workflow and is still just papering over the underlying problem -- which should really be solved inside t.d.a itself.

(monorepo is only a special case of the ask that I posted)

I’d be interested in hearing what you feel is clunky about it. Unifying all subrepo deps and starting a clojure program with that unified config seems less clunky (imo) than overriding the cross-user project config and having to put all user dev related aliases into git history. i would have waited for this to be addressed in tools.deps but from slack searches, this issue has been present for over 2 years and only suggested solution in convos has been overriding the CLJ_CONFIG value, or using Lein. So I figured some experimentation in user space would be useful.

"having to put all user dev related aliases into git history" -- huh? No, no one is suggesting that.

I think you're right: it solves a problem users are experiencing now. Eventually this library might become obsolete, but in the meanwhile, I think some people may benefit from it.

Unless you mean "common tooling for the monorepo" -- in which case, yes, that sort of stuff should be under git.

@UPH6EL9DH Maybe I'm reading your project wrong, but it looks like you would need to run that preprocessing step to create the actual deps.edn file after any changes?

@U04V70XH6 I think what he's doing is spitting out a deps.edn at CLI invocation time and then your script invokes clojure using that deps.edn: https://github.com/rejoice-cljc/interdep/blob/c248b47d9e4a6f45982cac41f2ab3990d6f5e310/example/scripts/run.clj#L60

so it writes on the fly, while you're invoking your commands

so the used deps.edn is ephemeral

which is a robust way of working around the current issues, since reading a single deps.edn is well supported

I understand that this is not the way things should end, but it's a nice temporary hacky workaround :)

> “having to put all user dev related aliases into git history” -- huh? No, no one is suggesting that. @U04V70XH6 I imagined that by overriding CLJ_CONFIG and not having a user cross project config, that devs would need to put their own custom configurations in git history. but perhaps you work around that, too, not sure.

@UPH6EL9DH I see your example script runs clojure in a .main subdir. Doesn't this cause issues with relative file paths in an app?

right now this is a simple library that provides utilities for processing nested deps.edn configurations, along with constraints that standardize the experience, document it, and make it more robust. it’s up to the user how they want to start their program, since there’s many different ways someone may want to do that. the way the example in the repo does it, is it calls the two helper utilities (`multi-repo/process-deps` and multi-alias/with-profiles) and those return a map with the values of the main deps config and the activated aliases. then it writes that deps config to a subdir and uses babaskha to start the clojure process from there https://github.com/rejoice-cljc/interdep/blob/c248b47d9e4a6f45982cac41f2ab3990d6f5e310/example/scripts/run.clj#L57-L72

I think it's better to start clojure from the directory you are in. So maybe it should support reading files that are not named deps.edn so it can spit out to deps.edn locally (because clojure doesn't currently have the option to point at another local deps2.edn file)

Note that https://github.com/borkdude/deps.clj (also works with babashka btw) supports -Sdeps-file. I'll end the hackathon discussion here. Going to bed now :).

> I see your example script runs `clojure` in a `.main` subdir. Doesn’t this cause issues with relative file paths in an app? @borkdude multi-repo/process-deps accepts :out-dir option that makes any paths relative to the desired directory but reading deps file from root directory is a good idea. I’ll see if that can be done with tools.deps; thanks for feedback, rest well!

@UPH6EL9DH > I imagined that by overriding CLJ_CONFIG and not having a user cross project config, that devs would need to put their own custom configurations in git history. but perhaps you work around that, too, not sure. Yeah, it's definitely a compromise. We have an agreed subset of tooling that's in the "master" deps.edn file, that satisfies everyone's needs for working on the monorepo. Luckily, we're all fans of fairly minimal tooling so we work with either plain REPLs or Socket REPLs. Mostly, the shared dev aliases are around testing and packaging (which of course we want standardized across the team, across all subprojects) and we have a :dev alias that points at a subproject in the monorepo that knows how to start various REPLs/tooling, based on whatever happens to be on your classpath (i.e., based on the set of aliases you used).

I think if we decided to switch to some sort of rewriting tool at startup, we'd want to keep it pretty simple: just slurping the "master" deps.edn file, reading it as EDN, sucking out a couple of the aliases, and just assoc'ing those into the local project deps.edn and writing it back out. FWIW, we already have a tool that merges all the subproject deps.edn files and writes to an everything pseudo-subproject that we can use as a convenient place to start a REPL if we want the entire codebase available (which is normally how I work). We manually build it, as needed, and we keep it under git. It still assumes the "master" deps.edn file is in play with its :defaults alias to specify :override-deps for everything.

This touches on how -X.. mvn-pom works, as well as other tools on https://github.com/clojure/tools.deps.alpha/wiki/Tools such as depot: the environment needed to run the tool is not the same as the environment the tool wants for analysis.

depstar "works around" this problem by relying on its own runtime environment being the same as what goes into the JAR file but just excluding itself. This is only tenable because it is a very small tool that has zero external dependencies. You can override this and tell depstar to use a different classpath, but that approach doesn't work for tools that actually want to look at the versions of dependencies, such as mvn-pom, depot, etc.

I do have this problem staked out in a table btw, just trying to decide on the right answer

not sure yet whether it should be a feature, a set of conventions, a set of conventions + code to use those conventions, etc.

Ah, OK. So no need for an "ask" on this one?

the datomic ion dev tools is another example

would be fine to have one, just by virtue of working on it for the datomic tools it's well on our radar already :)

OK, I'll add a question for it then...

> depstar "works around" this problem by relying on its own runtime environment being the same as what goes into the JAR file but just excluding itself. I also have a couple of tools that just receive a --classpath arg. But it depends on the tool. For mvn pom that might not be so convenient, and getting a list of aliases to resolve versions with might be better.

replied there 😄

@U47G49KHQ Can you elaborate on how that would help with invoking mvn-pom?

not sure that's what was asked.. did I read it wrong?

The question is how can tools like mvn-pom be made aware of aliases that should be considered in adjunction to the local deps.edn

ah, well... tools like mvn-pom should accept basis as argument instead of a list of aliases. Converting aliases to basis should be the outer-most place in such tools.

@U47G49KHQ how do you pass a "basis" from the command line?

you don't, that's why you need to use t.d.a as a library

That's not a useful answer to this question.

I mean, yes, of course the tool has to use t.d.a but the point of this is to standardize an API so that all tooling can work from a set of aliases without having to reinvent the wheel and rely on knowing so much about how t.d.a works under the hood.

And this also builds on the other question I posted on ask.clojure which allows for additional deps.edn files to be included in the process.

We don't want every single tool out there to have to write that boilerplate code.

here is hoping for tools.build 😄

That still wouldn't address either of these two issues.

(and we also wouldn't want to tie arbitrary tooling to tools.build usage either)

I have a feeling it is possible to build a variation of tools-deps that accepts multiple deps.edn files and configuration with aliases as args using tools.deps... as a library, that is then used with -X

These two issues both need to be solved inside t.d.a really.

Since -X mvn-pom needs a standardized way to do this, the solution will essentially be right there inside t.d.a -- once Alex et al have figured out the best way to solve it. I created the ask as a public placeholder for something that is already on Alex's plate, in a table of pros/cons for various solutions.

I think my suggestion answers your question 😕 There is a problem with mvn-pom , but any tool built on deps.edn can already build a basis given a set of aliases — which is what you asked

Your suggestion is actually the problem that I want solved. It isn't a solution 🙂

It’s not just aliases, there are other relevant params like -Srepro

We don't want every tool to have to write the same code over and over.

Yes, incorporating how -Srepro affects the merging of deps.edn would be important too. I don't use that flag right now because of course we have to rely on CLJ_CONFIG to "insert" that extra deps.edn file, which is why I didn't think to include it in the question 😐

I've updated https://ask.clojure.org/index.php/9848/tooling-construct-specified-aliases-without-boilerplate to try to make it clearer what is being asked here -- and also to include the point that Alex reminded me of, regarding -Srepro.

Is there an example of what that first bullet means that you can link me to (or easily describe)?

I'm guessing that it means that when specifying a jvm-specific dependency, that now works. @U47G49KHQ :)

huh, me?

@U47G49KHQ didn't you ask for this in relation to jfx deps?

well, cljfx uses activation profiles, but it always worked ¯\(ツ)/¯

Oh. So now I'm extra confused about what's new!

Updated. Will let you know if I run into any issues (but I doubt either of those changes will affect our workflow).

See https://ask.clojure.org/index.php/9847/how-do-i-resolve-maven-dependencies-for-a-local-jar for an example failure case

Wouldn’t be an issue with maven deps, only with local

@U064X3EF3 I grabbed the pre-release on my Linux box and gave it a run. Fixes the problem with the local .jar + maven deps. Everything else is running smoothly. Much gratitude!

I just released it as an official release, same bits so no need to re-install