Porting tools.deps.edn, I'm having a bit of trouble understanding the intention of root-deps:
(defn root-deps
"Read the root deps.edn resource from the classpath at the path
clojure/tools/deps/deps.edn"
[]
(let [url (jio/resource "clojure/tools/deps/deps.edn")]
(read-edn (BufferedReader. (InputStreamReader. (.openStream url))))))
I do understand that it will find a resource and read it. My question is the search path for that resrouce. jio/resource wraps ClassLoader.getResource , which I am led to believe searches the whole classpath looking for the resource.
The source for tools.deps.edn does contain a resources/clojure/tools.deps/deps.edn file. Is the intention that that specific file is to be read? Is that guaranteed to be found first, or could there be an override because of the order of searching? (which I assume is in classpath order, but I'm not sure what the classpath might look like in practice.)
Obvously I'm ignorant of both the intention of the code and the exact behavior of ClassLoader.getResource . Any insight appreciated.Yes, to all that. Resources in the jar are on the class path, and the root deps.edn that is delivered is the one inside tools.deps.edn. Not important at all that it's a resource, only that you can deliver that content to users of the lib.
For example, it could also be a hardcoded string and serve its purpose
Thanks for the clarification. Hard-coded the map directly since the file is just passed to read-edn.
(and, perhaps worth noting, the Clojure CLI install contains a copy of that deps.edn at the top of the installed tree, for legacy purposes, even tho' it is no longer used by the installation since that uses the resources version inside tools.deps itself, right @alexmiller?)
Yes but it is thus irrelevant for this conversation
Question about aliases and precedence: If I have a user-level deps.edn file and it contains an alias :foo, and then I specify -Sdeps {:aliases {:foo ...}} -M:foo when running the CLI, which :foo should I expect to get? Are they merged?
The reason I ask:
I have :attach in my user deps.edn (along with :repl) per the new CLI REPL, and then I start the -M:repl. I get a warning on a modern JVM about unnamed access from jLine. If I add the :jvm-opts to :attach to make that go away, it actually works—despite the CLI REPL code providing the :attach alias via -Sdeps without that :jvm-opts
Which seems to suggest that either the user level alias takes precedence (which seems unlikely/dangerous) or the aliases are merged (which also seems a bit odd).
Based on my quick experiments, it does seem that the alias bodies are merged?
They are merged per the rules here https://clojure.org/reference/clojure_cli#aliases
jvm-opts concatenate
I knew that for multiple different aliases being used together... but I guess I did not suspect that for multiple instances of the same alias across root, user, project, and "extra"...
For some reason, I assumed that if you provided the same alias in different "tiers", the last one won completely...
Oh right, sorry. That's a merge-with merge so generally replaces
However it may be that you're getting both specifically for jvm-opts because of how it comes through the cache. I dunno, could be a bug possibly
File an ask if you want me to look at it
I ran an experiment which confirmed that the same alias across multiple tiers is definitely merged per those rules. I was just surprised.
I changed my user-level alias to have
:main-opts that printed a message and verified that worked, then ran the CLI REPL and its :main-opts definitely overrode mine, but it still picked up my :jvm-opts.It's useful that it behaves that way but had initially confused me because I wasn't think of the merge. I'll give it some more thought.
But it does mean that Clojure tooling that runs clojure as a subprocess and provides its own aliases to use directly, can be affected by existing user- and project-level aliases which might be surprising to some folks. I wonder if there are any security implications there?
Huh... I ran another experiment and cannot repro: same alias just replaces across tiers... So maybe it is something in caching...
Got it! The merge of the tiers seems to apply a different merge strategy than the merge of aliases when used together. Across tiers, if you have the same alias in multiple tiers, the alias's data (map in this case) is just plain old merged. So, if user, project, and extra all provide
:jvm-opts then extra's version wins. However, if user provides :jvm-opts and neither project nor extra do, you still get the user-level :jvm-opts, even tho' you get the extra alias's other keys.In other words, if you have the same alias in multiple tiers and they specify some keys in common, it's a regular
merge. If you use different aliases in a CLI invocation, those are merged with the documented strategy, and :jvm-opts would be concatenated.So, specifying an alias in -Sdeps and using it in the invocation, can behave differently based on whether that alias appears in any of the root, user, or project deps.edn files.
https://clojure.org/reference/clojure_cli#deps_sources "The merge is essentially merge-with merge, except for :paths where only the last deps source :paths is used." -- although neither that nor the actual code (in tools.deps.edn) seems to support the behavior I'm actually seeing. Ugh! Okay, I'll need to write some specific tests against tools.deps.edn I think...
https://github.com/clojure/tools.deps/blob/dec86fcce4787c0a778fee673bf1e9e03b1a304a/src/main/clojure/clojure/tools/deps.clj#L702-L705 -- create-basis takes the EDN maps from all four tiers, gets the :aliases entry and does merge-with merge on those. That is why I'm seeing the behavior I'm seeing!
Although the EDN files themselves are "essentially merge-with merge'd", there is an additional layer happening with :aliases specifically when a basis is created, which I don't think is documented anywhere?
(sorry for all the noise in getting to this conclusion!)
Can you say more? The docs are trying to describe the code you're referring to (perhaps either docs or code is incorrect) but what is the additional layer you're referring to?
The docs describe something different. The docs are accurate about what they cover. This is a third behavior that is not covered in the docs. I'm writing a (long) Ask about it.
Oh, the merge-with is applied at the aliases level (one level further down) than described
So should really just be merge
That change might break existing "aliases as data" usage tho'?
And it would also break my current solution to https://github.com/clojure/clojure-cli.repl/issues/1 😄
(that it is a solution at all is due to this "newly-discovered" quirk)
It might
Since it's buried way down in that thread, I thought I should post this in the channel and see if anyone out there is relying on this behavior: https://ask.clojure.org/index.php/15242/behavior-when-alias-defined-multiple-deps-might-surprising (I suspect more people would be surprised by it than relying on it)