This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-10-22
Channels
- # 100-days-of-code (3)
- # announcements (7)
- # beginners (147)
- # cider (22)
- # cljdoc (24)
- # cljs-dev (71)
- # cljsrn (8)
- # clojars (3)
- # clojure (45)
- # clojure-conj (11)
- # clojure-dev (1)
- # clojure-italy (21)
- # clojure-nl (2)
- # clojure-spec (76)
- # clojure-sweden (2)
- # clojure-uk (100)
- # clojurebridge (3)
- # clojurescript (15)
- # cursive (7)
- # data-science (2)
- # datomic (7)
- # emacs (9)
- # events (2)
- # figwheel-main (4)
- # fulcro (117)
- # jobs (2)
- # jobs-discuss (21)
- # leiningen (184)
- # nyc (4)
- # off-topic (50)
- # planck (6)
- # re-frame (14)
- # reagent (25)
- # ring-swagger (5)
- # shadow-cljs (96)
- # spacemacs (5)
- # sql (26)
- # tools-deps (12)
- # uncomplicate (1)
- # yada (3)
G'day folks! I'm trying to configure a project to deploy two JVM-version-specific artifacts for the same version, but I'm running headlong into my shallow Leiningen knowledge. Apologies in advance for the following Wall-O-Text™.
Background:
* The project has different dependencies, depending on which JVM it's on - I currently handle these via :profiles
in the project.clj [1].
Objectives:
1. To deploy two "editions" of the library's JAR and POM (since the dependencies are different) to Clojars for each release.
2. To make it as easy as possible for consumers to select which "edition" of the library they want to use (e.g. via a :classifier
[2] - that seems like an elegant way to handle this).
Challenges:
1. Each of the JVM-version-specific artifacts has to be built on the JVM it's targeting - this is required so that the CI environment can run unit tests first, prior to deployment (the tests won't pass if they're run on the wrong JVM).
2. This implies that there will need to be two independent deployments to Clojars, for the same version of the library (albeit with different classifiers or whatever). I haven't been able to find out whether Clojars allows this kind of thing or not - I know it doesn't normally allow redeployment of non-SNAPSHOT versions though, which makes me pessimistic.
Learnings to date (take with a grain of salt - may be wrong), and some questions:
1. At least some Java libraries achieve this by twizzling their version strings directly (e.g. Google Guava [3][4]). Does Leiningen allow a project's version number to be overridden elsewhere in a defproject
form?
2. :classifiers
don't seem to be overridable in a :profile
(bug?), which makes me wonder if they're the right tool for this job. Or perhaps :profiles
is the wrong tool for this job?
3. The pom
task doesn't seem to take :classifiers
into account - it always generates an unversioned pom.xml
in the root of the project. Unless the deploy
task does some additional magic, this seems like it'll be problematic (i.e. cause clashes at deployment time).
4. The :jar-name
[5] property gets me halfway there (let's me set the JAR name on a profile-by-profile basis), but again I need both the JAR and the POM to be versioned, due to the dependency differences.
Any thoughts / suggestions?
[1] https://github.com/fn-fx/fn-fx/blob/master/project.clj
[2] https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L504-L510
[3] https://repo1.maven.org/maven2/com/google/guava/guava/
[4] https://github.com/google/guava/blob/master/android/pom.xml#L14
[5] https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L405-L407
Clojars shouldn’t (hopefully) have any problem with this - as a repo handling standard Maven repository format, this is not unusual. All of this is supportable in a Maven pom, usually with the assembly plugin (FYI). Not sure about any of the lein-specific stuff.
@deactivateduser10790 > 2. This implies that there will need to be two independent deployments to Clojars, for the same version of the library (albeit with different classifiers or whatever). I haven’t been able to find out whether Clojars allows this kind of thing or not - I know it doesn’t normally allow redeployment of non-SNAPSHOT versions though, which makes me pessimistic. why does it imply that though?
Could you not create each artifact with their respective jvm version, placing both into a :target-dir
and then do a deploy
of all those things after the fact?
Because each artifact comes from a different JVM and Leiningen can’t easily switch JVMs mid stream.
@deactivateduser10790 https://github.com/technomancy/leiningen/blob/stable/sample.project.clj#L282
Sure, but this doesn’t let me run the build with one JVM, then switch to a different JVM and run the build again, then finally combine the results (on either JVM) and deploy.
How does each artifact’s version string then get updated with the right classifier, given I’ve found the :classifier doesn’t get overridden within a profile?
and had 2 :aliases
:aliases {"jdk7stuff" ["do"
["with-profiles" "jdk7setup" <etc>]]
"jdk8stuff" ["do"
["with-profiles" "jdk8setup" <etc>]]}
can’t do that?Also, that breaks Travis’ (nice!) per-JVM job parallelisation. Not a showstopper mind you, but a loss nonetheless.
except your classifier problem, but I’m not aware of the issue you are referring to there
From my original post:
> 2. :classifiers
don't seem to be overridable in a :profile
(bug?), which makes me wonder if they're the right tool for this job. Or perhaps :profiles
is the wrong tool for this job?
but just making a target artifact name that reflects the classifier in it, but not sure that is great
Yeah I tried that, but can’t get two POMs out the backend.
Only two JARs.
The pom task doesn’t seem to respect :classifiers, and I haven’t found a way to tell it to use a different name (or directory).
I haven’t tried that, mostly because the docs for classifiers are so weak I didn’t even think of it. I’ll give it a try now.
They should - after all the Maven concept of a classifier is that it’s part of the version.
So different “versions” (different only in their classifier component) should be able to have different poms.
And I need that, as the dependencies are different between the two artifacts.
It's external to the POM
It's handled via the directory structure. See Google Guava for an example: https://repo1.maven.org/maven2/com/google/guava/guava/
(albeit they achieve their "jre" vs "android" editions by messing with the version numbers directly, possibly because of some of the same issues I'm facing)
At the end of the day, I just want to deploy something just like Google Guava, from a single project.
One project, two emitted artifacts, ideally differentiated by something in the version number.
Interestingly, for now the JAR files themselves are in fact identical - it's only the POMs that are different. But that may change down the line.
so if the pom isn’t different between the two, not sure why you think the pom
task should respect the classifier at all - doesn’t affect it
The pom IS different.
In fact that (for now) is the only difference between my two "editions" of the artifact.
(the code is identical, the dependencies are not)
Forget classifiers for a second - they're potentially part of the solution, but don't help explain the problem, and I don't know if I've expressed the problem very well yet. 😉
Forget also the consumption-side use of :classifier
- I'd like to be able to offer that UX to consumers, but if it makes the production side too hard then I'm willing to ditch it for equivalently easy methods (e.g. proprietary version strings, or even different artifact ids, if need be).
My issue right now is that I can't even produce the two artifacts I need.
And consumption-side UX is moot until I get to that point. 😉
My theory is that you may be able to get multiple lein tasks to run, that produce artifacts to the same :target-dir
and each having a name that represents its classifier. Once there, you could do a deploy after those tasks are done with their own Jvms.
aether
(which lein uses), does allow you to deploy
multiple artifacts and will respect classifiers.
yeah, not sure you’ll love the solutions to it if you are wanting something super dead simple
I'm ok with complex on the production side. Less so on the consumption side.
I think you are going to have to run multiple lein tasks that output to perhaps separate directories
I might as well just use bash for all of that. I suspect leiningen + plugins will only make that difficult.
(which is also what I was hoping to avoid, since it causes friction with TravisCI)
Yeah I noticed that. Kind of a pain.
Yeah no uberjars here - just regular JARs and their (different!) POMs.
That doesn't solve the POM collision though.
I still need to have two POMs and two JARs, and haven't found a way to get the pom
and/or deploy
plugins to do that.
I don’t know how to get your pom’s different, but I’m just assuming you can get :classifiers
to write the artifact names correctly so that it deploys with the correct classified name
I don’t know a great way of elegantly specifying files yourself directly to the deploy
tasks though
because it takes more args that you’d be repeating from the project, maybe works to do it though
Yeah though now I'm back to the "I need to switch JVMs between classifiers" problem.
Sadly this code won't work if the dependencies & JVM versions don't exactly match.
:profiles {:jdk-x-stuff {:java-cmd "jdk-x"}
:jdk-y-stuff {:java-cmd "jdk-y"}}
:classifiers {"jdk-x" :jdk-x-stuff
"jdk-y" :jdk-y-stuff}
Right, but that doesn't generate two separate poms.
It's not...
Recall that the dependencies are different
On JVM 1.8 the classes my library uses are part of the JDK. On JVM 11+ they're provided by a separate library.
Which I need to include in my library's dependencies, or else JVM 11 users of my library will be rather sad.
How do I generate two poms, one for jdk-x and one for jdk-y?
Yep. 😉
MY library has "variable" dependencies, depending on JVM version.
So I want to ship / emit / deploy two different "editions" of my library, ideally with the same version number, but with different classifiers (or whatever), so consumers can easily choose between them.
Right, and you can see how Google Guava does it.
They "stuff" the version number.
Same version number, but different classifiers for JRE vs Android.
Not really - Maven just thinks it's a classifier (since those have a fixed position in the version number, but other than that are just a freeform string value).
Right, but I need to twizzle the version # in order to get directories.
Naming isn't enough.
These things have to go into different directories (again look at how Google Guava does it: https://repo1.maven.org/maven2/com/google/guava/guava/ ),.
For every "version" there are two directories: 1. major.minor-jre 2. major.minor-android
Actually Maven will consider "jre" and "android" to be a classifier.
Those can be anything you want (with a few "special" values that have meaning to Maven for the purposes of version sorting).
Again not really - that's why Maven has classifiers.
> classifier: The classifier distinguishes artifacts that were built from the same POM but differ in content. It is some optional and arbitrary string that - if present - is appended to the artifact name just after the version number. As a motivation for this element, consider for example a project that offers an artifact targeting JRE 1.5 but at the same time also an artifact that still supports JRE 1.4. The first artifact could be equipped with the classifier jdk15 and the second one with jdk14 such that clients can choose which one to use. Another common use case for classifiers is to attach secondary artifacts to the project’s main artifact. If you browse the Maven central repository, you will notice that the classifiers sources and javadoc are used to deploy the project source code and API docs along with the packaged class files.
To give an escape hatch for libraries that have a single logical version with multiple different artifacts.
Right. The JDK14 / JDK15 example is a good one, if you imagine that a 3rd party dependency is needed to support JDK14, but not JDK15 (or vice versa),.
That's basically exactly my situation.
" The classifier distinguishes artifacts that were built from the same POM but differ in content.”
Right. And what I figured out is that they mess with the version number during the build, so effectively they provide multiple POMs.
Question is - does Leiningen allow the version number to be messed with, ideally inside a profile.
Cause then my problem would be solved. 😉
well, I guess I’m wrong somewhat, you can do different bulid steps with deps in the same pom, so nvm
I see bash in my immediate future...
And TravisCI pain...
Thanks for talking this through with me though - really appreciate it!
Glad I'm not just missing something obvious.
In the project.clj you mean, or post-process the generated pom.xml?
:profiles {:jdk-x-stuff {:version "1.0-jdk-x"
:java-cmd "jdk-x"}
:jdk-y-stuff {:version "1.0-jdk-y"
:java-cmd "jdk-y"}}
but when dealing with the project data in like a plugin, the version is just under the key :version
For anyone following along at home, here's where I eventually landed: https://github.com/fn-fx/fn-fx/blob/c3eb3eb07eaa4bdb0f18724f08df7596da69c8ce/project.clj #dirtyhacks