This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-10-11
Channels
- # announcements (1)
- # aws (2)
- # beginners (43)
- # calva (7)
- # cider (5)
- # cljs-dev (1)
- # cljsrn (2)
- # clojure (68)
- # clojure-android (10)
- # clojure-conj (2)
- # clojure-italy (10)
- # clojure-nl (3)
- # clojure-spec (3)
- # clojure-uk (128)
- # clojurescript (10)
- # core-async (47)
- # cursive (32)
- # datomic (35)
- # events (1)
- # fulcro (24)
- # funcool (2)
- # graphql (2)
- # jobs (3)
- # juxt (2)
- # leiningen (3)
- # luminus (1)
- # off-topic (7)
- # om (1)
- # onyx (2)
- # pedestal (32)
- # perun (2)
- # portkey (2)
- # re-frame (4)
- # ring-swagger (2)
- # rum (3)
- # shadow-cljs (137)
- # spacemacs (4)
- # testing (3)
- # tools-deps (101)
- # uncomplicate (2)
- # unrepl (5)
- # vim (2)
I needed very simple uberjar functionality with clojure.main
as the Main-Class
in the manifest. I forked and updated depstar to provide that https://github.com/seancorfield/depstar/commit/b49f15b0fc341f3004cb1bbf037334e3acbcb900 (a PR is in to @ghadi but my last PR to update the readme to explain the lack of manifest hasn't garnered a response yet so...).
;; uberjar building:
;; - see for a minimalist approach:
;; - uses my fork that adds manifest support
;; - clj -A:depstar result.jar -m
;; - sets clojure.main-main as the entry point
;; - clj -A:depstar result.jar my.project
;; - sets my.project/-main as the entry point
:depstar {:extra-deps
{com.healthfinch/depstar
{:git/url ""
:sha "a328136da59ea9b83483189451fb71aafd13bdb1"}}
:main-opts ["-m" "hf.depstar.uberjar"]}
As a beginner to tools.deps and Clojure in general, I just banged my head against the wall for a couple hours figuring this implementation detail out, because the documentation leaves off this edge case. I discovered something with alias loading I thought would be worthwhile to share here in any case:
Assuming you have 2 aliases with :main-opts
keys, regardless if other keys from other alias types are present in them or not:
If -M
is specified on the command line for both aliases, only the last alias is used as documented.
If -A
is specified on the command line for both aliases, only the last alias is used as per the documentation for -M
.
If -A
is specified for one, and then -M
is specified for the other, only the last one is used.
If -M
is specified for one, and then -A
is specified for the other, still only the -M
one is used, even though -A
is later.
It's that last rule that got me good. In short, you have to be very careful here.
Anyone recommendations for a thing that will package my deps.edn lib into a jar and install it into the local repo?
Pack doesn't do regular jars, cambada doesn't install them into local maven, meyvn seems a little more involved
personnaly I am half tempted to just rely on having clj installed and run apps with it, more or less what you do with cljdocs I think @martinklepsch right (bash script that downloads the archive, upacks it and runs)?
capsule based build seemed quite a good idea (one of the options on juxt.pack), but I am not sure where that project is heading
I'm actually referring to regular jars (not uberjars)... e.g. when publishing a library
@martinklepsch fwiw, you could use pack with skinny jar mode and write a pom.xml. But you'd have to install that jar yourself (with maven I guess?)
@dominicm nothing special in mind. pack is imho the most complete solution so far. One minor missing piece would be the ability to specify a main on the one-jar option, but i can live without it
There should be an issue open for this. It is something I want to fix. I don't want to require AOT from users, so I want to patch onejar a little to support a default param to clojure.main.
Would be nice if there was only one well supported solution for packaging. I stared with cambada but had to fork to apply some PRs
@orestis sounds like me forking depstar to apply a PR to add a minimal manifest 😀
@martinklepsch is there a reason you need a jar in your local repo? Can't you refer to that local library source via :local/root?
@seancorfield I can but I have some wierd interdependencies that aren't supported fully by Cursive/tools.deps
Right, transient :local/root
deps are not well supported (by deps itself, and also Cursive - I’m fixing that right now)
Even though I don’t use Clojure professionally yet, I’d chip in for an Arachne-like Kickstarter, but for a new build tool based on tools.deps and simple libraries.
There’s so much duplication of effort these days - jar building, linters, formatters and so on.
I have a dream for a build tool that would just provide an (overridable/extendable) set of library decisions and some glue code to invoke them from the command line.
What you are describing is probably a program 😕
But yeah, there is a lot of churn in the dev space right now
Will probably be a while before it simmers down with some kind of best practices
The whole point is to rally everyone around a single tool. I think this has multiplier benefits for the community and newcomers. See for example Elixir and their mix tool.
There is certainly value for newcomers.
Atm Lein is probably good for them
But tools-deps is the first system that let me understand wtf was going on when starting up
I've been recommending clj
to newcomers -- with the caveat that most books/tutorials use lein
. At least with clj
, you can point newcomers at "official" documentation in a central place and you know the tooling is both simple and well-supported.
For a long time, at work we've done no AOT and we build over uberjars to have clojure.main
as an entry point. Our needs are very simple. We used to run a lot of stuff from source directly via lein
and later boot
but I think packaging as an uberjar for java -jar
is simpler and safer (no need to fetch dependencies onto the target systems).
+1 for building uberjars
ClojureScript has had years of churn but seems to be settling down now. Maybe Clojure needs a bit of churn to shake things out and give us better, simpler tools too?
tools-deps is a great step in that direction. But causes a bit of chaos in the transition
To be clear, IMO tools-deps enables the creation of such a mostly-glue tool. Otherwise making a new build tool from scratch is a huge undertaking.
@martinklepsch the official story there seems to be write a pom.xml yourself, use the -Spom
task to keep the dep lists in sync? (I don’t really understand what it does) and mvn install yourself.
Having attempted to write that build tool, I'm not happy with anything I've come up with. Build tooling is hard to get right, especially around tools like figwheel.
Krei.alpha and kick.alpha are the two attempts by JUXT. We also did mach in the past which we used for alternative purposes.
@U064X3EF3 keeps mentioning that builds are programs, so I suspect that both of those attempts to be data driven are fatally flawed. I've been thinking of implement a paper by peyton jones for creating build systems a la carte, maybe that will unlock the pieces needed, but I'm not sure yet.
My naive attempt so far is to just invoke some namespaces - either external or my own via aliases. My need is for more well rounded and robust libraries to do things like jars and so on.
Caching. Minimal builds. Early cut off. These are the general problems to solve. Cloud building is a nice to have in some contexts, but none I'm in.
I don’t think that there is any way around the current slow startup of clojure, is it? Even with precompiled stuff, you will get at least a handful of seconds which is already too much. Hard stuff.
you could always fire up a build-tooling repl once, that puts you in a namespace with functions like (build)
(test)
(do-everything)
Something quite hard to build at a REPL is AOT-compiling namespaces. You have to start a new JVM to do that. You might be able to get away with something like a pod, but that's the best I have. Just as a result of how it decides to cache/skip namespaces.
To be honest, I think you have to just make it a library that you can run from the repl. It has to happen around the Clojure command line tools, else there is too much friction.
Even shell scripts get in the way of tooling. Cursive won't work if you require funk.
I really do like depstar's minimalism but it's frustrating that it omitted a minimal manifest -- and I'm not sure why it did that. Everything else in the build space is much more complex 😐
@seancorfield the easy answer is that generating manifests is out of scope. War is another format that needs a manifest to be written. Simpler to let users decide how to handle extra files in their classpath.
yes, not so minimal when you start having to do that…
Adding a manifest with just a main class is only a few lines of Clojure when you're building a JAR tho'... 🙂
But it conflicts with any additional things a user may want to insert into a manifest themselves. Users can also generate a manifest in a few lines of code. In most cases of can just be hardcoded by the user.
Right, which is why it's important that adding the minimal manifest is optional. depstar specifically excludes META-INF/*.MF
files from the classpath so the resulting JAR is guaranteed to have no manifest. My fork allows users to conditionally add a minimal manifest that specifies Main-Class
(only). That seems to be such a common need -- for java -jar ...
-- that baking it in as an option seems worthwhile.
The code is based on Boot's create-manifest
so it could also take a hash map of key/value pairs to add as extra manifest attributes, but that's not exposed to the command-line in my fork (yet).
I see. That seems quite wrong. It also removes license files which is not allowed under some licenses like Apache
Yeah, I noticed that and thought it was weird. I don't know why it does that (and Ghadi hasn't responded to any of my questions about it yet).
We're not redistributing our JARs so the license issue isn't a problem -- and depstar filters out a lot of other stuff too so it's pretty brutal in that respect.
Is there an algorithmic reason beyond wanting to track the path for every expanded dep that expand-deps
couldn’t group deps into resolution batches by manifest type? The Maven implementation used to read manifests seems designed for reading many manifests in one go, and less so for reading a single manifest at a time.
The hard thing is proving that letting Maven potentially solve range constraints instead of using deps’ behavior for everything can’t produce a different final dependency solution 😕
Maven will produce a different solution, and we don’t want it
Ah. Yeah if you have a non-Maven pinning (or even just winning coordinate) for a dep, and Maven resolves something else, you can’t really back the transitive consequences of that out of Maven’s solution so you loose minimality for sure which can have other consequences.
Regarding my message above about aliases, does anyone think this deserves an issue opened to at the very least document this edge case?
The order independence of -M
winning when -A
and -M
is present is probably worth documenting.
I would prefer to see it fixed, but documentation would be an improvement over the current situation.
Personally I'd like to see the behavior change. The reason why is this:
: I have a :dev alias that is only ever project-local. I use it for example to require my namespace when starting up an nrepl in cider. This requires a :main-opts key, making it of the -M type, but as per the documentation, only the last -M type is ever used. I also want to compose this with a debugging alias to say depend on and require spyscope with another alias. Currently this is impossible without putting that in the :dev alias and thus copying it to be project-local instead of a global alias.
Unless there is some other way to compose a global main-opts with a project local one that I do not know of
Maybe I'm just asking too much from this too new tool?
FWIW, I only ever use -A
and I split aliases out so I run into this issue (downside: you need to specify multiple aliases for some things).
Perhaps @U064X3EF3 can weigh in here as I see he is the project lead. When i get home later I'll open an issue if only to document this edge case so other newcomers don't bang their heads like I did.
It is better to ask about whether it can change when things are still alpha, than later. It is really up to folks like Alex, Rich, and Stu Halloway on how they choose to make the tool work, so don't be surprised with the answer "doc enhancement, please"
My sense is that it is "by design" that aliases are combined in order, left-to-right and that -A:foo
is just shorthand for -R:foo -C:foo -M:foo ...
-- so the behavior you observe is intentional and likely considered to be already well-documented.
If you specify your project-local alias with -R
and/or -C
instead of -A
then you'll get the behavior you want.
The confusing bit is that -M
is documented as only the last one being used, but if -A
is indeed shorthand for that, an -A
after an -M
needs to have a note mentioning that only the last explicit -M
is applied, and -A
is ignored.
The -A
is not ignored -- the aliases are all combined left-to-right, but with :main-opts
it's always last one wins.
But that is not the case.
-M:foo -A:bar
, when both foo and bar have :main-opts
keys, then foo
wins.
Ah, gotcha. Here's why:
val="$(join '' ${resolve_aliases[@]})|$(join '' ${classpath_aliases[@]})|$(join '' ${all_aliases[@]})|$(join '' ${jvm_aliases[@]})|$(join '' ${main_aliases[@]})|$deps_data"
So -M
is always applied last. OK, yes, that should be documented. Do you want to create a JIRA ticket for that (against the CLJ JIRA project I believe)?
I'm happy to create the ticket if you want me to.
I very much want to so others don't run into the headaches I did. I won't be able to do that until tonight or tomorrow though when I'm at my dev machine, unless you don't mind.
I'll go ahead and create it... against the website since it's really an error in the Reference page for the CLI
This is a bug, not a doc issue. Order matters, should be respected.
OK, good to know. So the shell script would need to expand each -A
as it encountered it, adding the aliases as if all four were really specified explicitly?
Do you want a JIRA issue (and maybe a patch) against CLJ for that instead?
Jira yes, patch if you like. I suspect it affects the scripts too so may require some coordination
Ugh! Yes, fixing the clojure
script would solve one side of things -- that's easy -- but tools.deps
also needs to be fixed too. OK. JIRA for now. I'll look at a patch if I get enough free time but it's a bigger change than I had initially thought.
Thank you everyone.