This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-07-10
Channels
- # announcements (3)
- # architecture (54)
- # babashka (11)
- # beginners (12)
- # calva (5)
- # clj-on-windows (1)
- # cljdoc (2)
- # cljs-dev (1)
- # cljsrn (6)
- # clojure (130)
- # clojure-europe (8)
- # clojurescript (21)
- # conjure (23)
- # core-async (4)
- # datomic (7)
- # depstar (77)
- # events (1)
- # fulcro (27)
- # lsp (88)
- # malli (5)
- # meander (1)
- # off-topic (4)
- # pathom (43)
- # polylith (39)
- # re-frame (9)
- # shadow-cljs (14)
- # timbre (3)
- # tools-deps (53)
(this may be unique to clj-new
because of the way it tries to dynamically resolve/load dependencies to find the specified template? the error EDN file does not mention the missing :sha
failure at all)
yeah, that's just the nature of it. you can still use :sha though
I think you're missing the point: :git/sha
is what is installed by the CLI, not clj-new
.
I think this will cause problems for any t.d.a-based tool that is installed via the new CLI but relies on an older t.d.a (that does not recognize :git/sha
) to create an environment for executing code?
The only difference between clj-new
1.1.314 which works and 1.1.309 which doesn't, is the version of t.d.a that they depend on.
(and the new release now has :tools/usage
in deps.edn
but that's not relevant for this)
doesn’t work at install time or at tool execution time?
I showed exactly what doesn't work in the main channel... Let me know if I need to explain it better but it should be an exact repro.
I have a kind of open ended question about deps.edn
and library/program design. Say I want to offer users of my library / program two different usage patterns that have different dependencies: the use case I'm thinking of here would be one version of a program that runs purely in memory and one that uses a database for persistence. For those users who want to run it only in memory, is there a way for them to pull in my library without pulling in the DB deps as well?
@afoltzm usually you make these dependencies optional, by just not making them dependencies of your lib
if you want to provide the storage-related functionality, you tell users to include extra deps
I guess I missed this, it seems the :
is now not used between the flags , or is the :
between the flag and first alias name optional (maybe it always was and I didnt notice it)?
clojure -X:project/new
clojure -Xproject/new
That's probably a clue to get the latest version and do some UX testing 🙂Aesthetically I like the initial colon, especially when using multiple aliases clojure -M:fig:cider-cljs
, feels more consistent. I appreciate I seem to be weird in that I like typing... :rolling_on_the_floor_laughing:
@jr0cket iirc, officially only -X:fig:cider-cljs
is supported, the other happens to work and the code was intentionally written to work with that kind of input.
tool names are symbols and should not use a colon (with -T
)
in some cases, aliases without colons are tolerated (but don't do that)
Ah, so T
is in the same style of flags as J
and S
it would seem. I should definitely get the latest and have a look at clojure --help
. Thanks all.
when you find it is lacking, please let me know :)
I do have some todos in the syntax documentation area for next week
In the clj -h
I think this could potentially be misleading, especially those new to Clojure CLI and the concept of aliases
exec-opts:
-Aaliases Use concatenated aliases to modify classpath
-X[aliases] Use concatenated aliases to modify classpath or supply exec fn/args
-M[aliases] Use concatenated aliases to modify classpath or supply main opts
-P Prepare deps - download libs, cache classpath, but don't exec
It assumes it is known that an alias is :new
and not just the name new
. It is stated that aliases are keywords earlier in the texts, but that can easily be missed
Perhaps adding a specific example as with -X:deps
section
Programs provided by :deps alias:
-X:deps mvn-install Install a maven jar to the local repository cache
-X:deps git-resolve-tags Resolve git coord tags to shas and update deps.edn
I have always found a few specific examples help clarify the general approachThe help here is something I have struggled to make simultaneously small and useful :)
@jr0cket Did you know clojure has a man page as well? :-) You might in fact be the person who mentioned it here, don't remember.
I did know, but wasnt the one who mentioned it. I will read it (when I've installed the new version)
In the clj -h
I think this could potentially be misleading, especially those new to Clojure CLI and the concept of aliases
exec-opts:
-Aaliases Use concatenated aliases to modify classpath
-X[aliases] Use concatenated aliases to modify classpath or supply exec fn/args
-M[aliases] Use concatenated aliases to modify classpath or supply main opts
-P Prepare deps - download libs, cache classpath, but don't exec
It assumes it is known that an alias is :new
and not just the name new
. It is stated that aliases are keywords earlier in the texts, but that can easily be missed
Perhaps adding a specific example as with -X:deps
section
Programs provided by :deps alias:
-X:deps mvn-install Install a maven jar to the local repository cache
-X:deps git-resolve-tags Resolve git coord tags to shas and update deps.edn
I have always found a few specific examples help clarify the general approachThe man clojure
page looks very nice and much clearer I believe, e.g
clj-opts
-Jopt Pass opt through in java_opts, ex: -J-Xmx512m
-A:alias...
Concatenated aliases for REPL execution, ex: -A:dev:mem
-X:alias... fn KPATH V
Exec alias to invoke a function that takes a map, with keypath/value overrides. Function must either be supplied or be in :exec-fn argmap for alias
-M:alias...
Exec clojure.main, either from opts in alias or command line
I am learning about how aliases can be used as replacement for the maven “provided” dependency scope and I am having trouble understanding how to accomplish a very specific kind of compilation task.
My AOT compiled application is deployed into an environment that supplies some pre-compiled deps. The application references classes in these pre-compiled deps and thus they must be on the compile-time classpath. However, I do no not want these deps (or their transitive deps) in the final uberjar.
I understand how aliases allow for these deps to only be provided during the “build” task, but I have not been able to figure out how to declare these deps as being compile-time only. My current understanding is that this is https://ask.clojure.org/index.php/9110/scope-in-deps-edn-should-be-added-and-not-deleted-from-pom-xml?show=9117#c9117 not supported by tools.deps
because aliases can be used to same the effect. However aliases don’t have any control over if a dep is included in an uberjar and it would be the responsibility of the assembly/build tool (for example https://github.com/seancorfield/depstar) to handle the exclusion of certain deps in the uberjar. Is this understanding correct?
In case it is helpful, my particular environment is a https://spark.apache.org/ that provides Spark as a pre-compiled dep at runtime.
I believe you are right with the build tool handling the exclusion of certain deps. It turns out that depstar has a way to specify specific namespaces/classes not be included in the final uberjar. You may look at that to solve this particular issue.
Thanks! I played around with the :exclude
option on depstar but (as far as I could tell) you would have to carefully identify all the unnecessary transitive dependancies and curate a set of regex that exclude them by name. I asked over in #depstar to see if I was missing something.
Spark is a use case that depstar is specifically designed to work with. That is why the compile alias stuff was added @U7XR2PZFW
I'm not at a computer to give you more details. You need to read the depstar docs closely about aliases.
This is the relevant issue @U7XR2PZFW https://github.com/seancorfield/depstar/issues/75
(so this was Storm, but the same thing applies for Spark, and the approach is to separate out "provided" deps into an alias so it doesn't become part of the JAR but specify that alias to the "compile" phase of depstar
so it is available for AOT compilation)
Thank you @U04V70XH6! I was indeed missing the :compile-aliases
option in depstar.
I’m still mentally decoupling the notion of “scopes” used in maven into the independent parts that we prefer in Clojure. Based on everyone’s responses it seems like my mental model is correct: It is not the role of deps.edn
to declare the separation between compile-time and runtime classpaths, but rather this is handled by an assembly/build tool (like depstar or tools.build). Furthermore these tools leverage aliases (which is a construct provided natively by deps.edn
) to compose a classpath.
This is clear to me now, and I like it a lot! Thanks all. Super helpful, as always. 🙂
I can't speak to depstar, but the new https://clojure.org/guides/tools_build library allows you to construct a "basis" (a classpath execution context) in your build with whatever modifications you need. Additionally, you can construct a different basis for different parts of your build. So for example you could define an alias with the compile-time library:
(def compile-basis (b/create-basis {:aliases [:compile-libs]}))
;; use that compile-basis with b/compile-clj
(def runtime-basis (b/create-basis {})) ;; default basis, matches project with no aliases
;; use runtime-basis to b/write-pom (declares your dependencies in the pom from the basis for publishing)
;; use runtime-basis with b/uber to build the uber jar
Thanks @U064X3EF3! I will spend some time learning tools.build. It sounds like it has the level of control that I require.
depstar
has options specifically for this use case (well, originally added to support Storm). I explained in the other thread.
the key here being that it's a program and you can do whatever you want :)
With the new tools.build stuff, has there been any improvement in constructing classpaths with deps that do not include Clojure, to allow it to be used for non-Clojure projects (or non-Clojure modules in mixed-language projects)? Last time I looked at this that was impossible since the system deps.edn always pulls it in and there was no way to override that.
:classpath-overrides {org.clojure/clojure ""
org.clojure/spec.alpha ""
org.clojure/core.specs.alpha ""}
No worries - that’s interesting, I’ll try that, thanks. I’m surprised that works, I must admit. I’d be interested in Alex’s view on whether that’s defined behaviour or a happy accident 🙂
@borkdude I guess that means you always build them using an alias like :no-clojure
or something similar, is that right?
@cfleming This is what's used in bb: https://github.com/babashka/babashka/blob/b6e65c08a8fef8d04d9ad46685a99b4902e52e89/src/babashka/impl/deps.clj#L54-L76
Thanks, yeah, I was just looking at your repo. I see you also have :classpath-overrides {org.clojure/clojure nil ... }
in one place, does that work similarly?