This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-09-10
Channels
- # announcements (16)
- # aws (1)
- # babashka (2)
- # beginners (33)
- # biff (2)
- # clj-kondo (13)
- # cljs-dev (1)
- # cljsrn (3)
- # clojars (6)
- # clojure (198)
- # clojure-australia (3)
- # clojure-europe (41)
- # clojure-france (3)
- # clojure-nl (2)
- # clojure-spec (7)
- # clojure-uk (12)
- # clojurescript (57)
- # clojureverse-ops (1)
- # code-reviews (3)
- # community-development (2)
- # conjure (10)
- # data-science (1)
- # datomic (15)
- # depstar (2)
- # docker (2)
- # etaoin (1)
- # events (1)
- # exercism (5)
- # fulcro (23)
- # helix (23)
- # introduce-yourself (4)
- # jobs (6)
- # kaocha (1)
- # lsp (11)
- # meander (107)
- # off-topic (8)
- # pathom (3)
- # polylith (33)
- # re-frame (23)
- # reagent (7)
- # reitit (28)
- # remote-jobs (3)
- # sci (1)
- # shadow-cljs (2)
- # specter (5)
- # sql (38)
- # tools-deps (72)
- # web-security (9)
- # xtdb (32)
Not 100% if this should be asked here or in #tools-build as they probably have some overlap, can I use deps to build a project comprised of modules like https://github.com/metosin/reitit?
Yes 🙂
IIRC juxt’s edge used to demonstrate this kind of setup: https://github.com/juxt/edge
I assume it still does; but it constantly evolves
See also @U04V70XH6’s blog posts on how to arrange a clj monorepo: https://corfield.org/blog/2021/02/23/deps-edn-monorepo/
I think the difference between Sean's use case and this one is how artifacts are built and distributed. When you have an internal monorepo for building deployable uberjars you don't really care about deploying your jars to some place like clojars, the builds are self contained. In this case I want to distribute the building blocks
Monorepo where the end product is deployable artifacts: I grok Monorepo where the end product is library modules with inter-dependencies: panik
Fair enough — to be fair I’d like to do that too… Unfortunately some of our components are both open and closed source so it’s not really viable in a mono repo.
Anyway I think tools-deps and tools-build or babashka tasks or a combination of them will give you much of what you need. Essentially all you need is a clj mono-repo, with a script that iterates over the modules and deploys them with depstar. Obviously assembling all that will involve building something somewhat bespoke… though you’d imagine it could be released as a clj build tool.
Yes.
I found when I started exploring this problem space that dependencies specified as taken from a :local/root
aren't taken into the pom, so I have to figure out something more clever
Yeah, maven has no way to talk about that (or git deps)
Should they be resolved to an absolute path relative to the project root at pom creation time?
"they" = ?
the problem is the pom has no way to talk about "local" artifacts, only things in Maven repos
What I mean is, can local/root deps be resolved to where they will be once the artifacts are built and referred to as such in the pom? That way consumers of the module will be able to pull in the dependencies.
Sort of like readlink
or dereferencing
"and referred to as such in the pom" -> no, there is no language for this in pom.xml
pom.xml specifies deps only as groupId/artifactId/version for artifacts found in maven repos
so you would need to at least "install" those artifacts to your local repo for them to be addressable by the pom.xml
I was thinking more along the lines of • I know at build time what the groupID/artifactID/version will be because I can get to the other repo by way of the local/root • Write those in the pom instead of install them locally • "eventually consistent" • ... • profit?
is something deploying to a maven repo in this process?
I mean, all of this seems like a bad idea if you're using Maven-oriented tools :)
Maven does have a whole thing (multi-module projects, pom inheritance, etc) for building and deploying a set of related projects from a single repo
but you need to be "doing Maven" for that whole thing to make sense
Brian Goetz likes to tell how people start questions with "why don't you just _" when they have a big ask, so I won't ask that, but I would say it would be nice to be able to do that, both for deployment (maven/clojars/artifactory) and for deps, to be able to pull in only parts of a code base which have already been organized as such Which reminds me of the spec-ulation talk, now
@UK0810AQ2 Have you looked at how Polylith structures things? It completely separates the deployable part from the code so you could have a "project" for each thing you want to deploy to Maven/Clojars and separate projects for what your internally deployable stuff is -- and then the code is "assembled" from bases (usually things with a -main
) and components (small "libraries").
As noted in this thread, if you build something that you deploy to Maven/Clojars, then it must declare all of its dependencies as things that are also on Maven/Clojars -- but with a structure like Polylith, you can choose to build different artifacts for different audiences, so you could use git deps with :deps/root
and :local/root
dependencies and that would work "internally". Polylith's poly
tool is a "project" inside the polyfy/polylith
monorepo and can be used as a git dep, but they also build other artifacts from it.
Yes, but as you noted, it does not solve the dependencies issue, and I wouldn't want to maintain two dependency maps. I would like to programmatically transform the "is" to "ought". It might be easier for maven artifacts than for deps, actually
Hmm, then I guess I don't understand what you're trying to do or why Polylith's approach won't solve your problem...
I don't use Leiningen so I don't really understand what's going on there -- but superficially it looks very, very similar to a Polylith structure with a workspace-level deps.edn
(`project.clj` in reitit) and then each project/base/component having its own deps.edn
(`project.clj` in reitit).
@UK0810AQ2 Author of edge, so another person very familiar with this space. Doing the reitit thing with deps.edn is very easy! Let's start with our basic layout: Foo-core/ Foo-plugin/ Each folder will need: • deps.edn • pom.xml (to hold version, etc.) You can see that reitit updates the version everywhere on release: https://github.com/metosin/reitit/commit/8694d312f86d14ac7df3fa431af7cfba6873d8d8#diff-498cb648cb6ba613422c242cc068100c0bafa70b5cc71193570205abdac73b9e but you can maybe automate this with some tooling! deps.edn really has 2 modes of operation here: release & development. During development, you're looking to depend on the local copies of dependencies (presumably), and during release you're looking to depend on maven central versions of the same version as your version. One way to manage this would be to create a parent pom which has the version in and sets the versions for all the modules in your project. Another option is just to manually maintain which version your module depends on by doing it as part of the update to the pom.xmls in each module.
Oh, so to do development in your deps.edn you should have a :dev
alias like so:
:dev {:override-deps {Foo-core/Foo-core {:local/root "../Foo-core"}}}
Can children poms derive the version from the parent? Wondering how this will look like with tools build, too
We use tools.build
with Polylith so I'd say that's completely orthogonal.
tools.build doesn't interact here, particularly. @U04V70XH6 was working on stuff to make builds in other projects work, but that's an optimization that applies fractionally to libraries.
There's certainly no conflict between having a unified :dev
experience with everything using :local/root
style deps and then building deployable artifacts that all have Maven-style coordinates and deps.
@UK0810AQ2 I believe a feature of maven called managed deps enables that, yes.
I think a question like this has come up before, and I'd like to do something like this for the juxt/yada repo if I get some time. Perhaps a small example repo would be worthwhile.
Is it expected behavior to get WARNING: Implicit use of clojure.main with options is deprecated, use -M
when running clj --repl
or clj -e '(+ 1 2)'
? I don't necessarily have any aliases to use in those contexts.
ohhh... interesting. I didn't even realize / try it w/o an alias. hence the bracketed [aliases]
in the help output. and the message literally tells me use -M
😆 thanks @U0NCTKEV8!
I believe it is, the idea being if you want to invoke clojure.main you need to pass -M
I'm trying to deploy to heroku but it seems they use Clojure 1.10.0.411
so they are not liking my up to date depstar invocation. I currently have this in my deps.edn
:
:aliases {:uberjar {:replace-deps {com.github.seancorfield/depstar {:mvn/version "2.1.297"}}
:exec-fn hf.depstar/uberjar
:exec-args {:aot true}}}
and was calling it with clojure -X:uberjar :jar my-app.jar
but it doesn't like -X
which I replaced with -A
, :replace-deps
had to be replaced with :extra-deps
and I'm at a loss with what to do for the :exec-fn ...
. What would you advise there?The version of Clojure CLI can be specified with the Heroku Config Car CLOJURE_CLI_VERSION
https://practical.li/clojure-web-services/projects/banking-on-clojure/deployment-via-ci.html#heroku-pipeline-configuration
Oh man, I wish I had found this earlier yesterday. That $PORT thing was driving me crazy before I discovered what I was doing wrong.
Petition Heroku to stop using such an ancient version of the CLI? 🙂 I'm a bit surprised you can't specify which CLI version you want, like you can for CircleCI or GitHub Actions...?
Given that, you'll need to switch to the old -M
invocation of depstar
(which is no longer documented!)...
You'll want :main-opts
in :uberjar
with ["-m" "hf.depstar.uberjar" "-C"]
plus whatever additional args you need @chase-lambert
The old -main
entry point doesn't support all the options that the -X
entry point does. Here's the excerpt that does the argument parsing:
("-C" "--compile") [{:aot true} (next args)]
("-P" "--classpath") [{:classpath (fnext args)} (nnext args)]
("-D" "--debug-clash") [{:debug-clash true} (next args)]
("-h" "--help") [{:help true} (next args)]
("-J" "--jar") [{:jar (fnext args)} (nnext args)]
("-m" "--main") [{:main-class (fnext args)} (nnext args)]
("-n" "--no-pom") [{:no-pom true} (next args)]
("-S" "--suppress-clash") [{:debug-clash false} (next args)]
("-v" "--verbose") [{:verbose true} (next args)]
("-X" "--exclude") [{:exclude (fnext args)} (nnext args)]
so you'll probably want --jar my-app.jar -m your-main-ns
maybe others?Yeah I wish the official heroku buildpack was more up to date. It seems they are closing the issues asking for an official deps.edn version. There are various workarounds and community built buildpacks floating out there
According to Heroku's own docs you can specify a more modern CLI via the CLOJURE_CLI_VERSION
environment variable.
See https://devcenter.heroku.com/articles/clojure-support#setting-clojure-cli-version
Awesome, I'll just find a way to set that variable then so I don't get future conflicts as I continue to add to my deps.edn
Here's the lines in their build pack that handles arbitrary CLI installation: https://github.com/heroku/heroku-buildpack-clojure/blob/main/bin/compile#L33-L37
Settting "config vars" (aka environment variables I assume): https://devcenter.heroku.com/articles/config-vars#managing-config-vars
(I'm guessing here, based on their docs -- I have never used Heroku)
NP. By learning a bit more about Heroku, I'm better able to help others who may have Qs about it, even if I don't use it myself 🙂
I used to work at Heroku, so am fairly familiar with there tooling. The buildpack is something I'd like to revisit as my current approach is not as smooth as it should be. If there isn't a suitable Heroku buildpack, then I'll consider creating a suitable community version.
Hello all. Happy user of deps.edn and depstar to build jars and uberjars - should I be thinking to replace my depstar usage with a tools.build build.edn? Any recommendations - I don’t do anything very complex.
@mark354 For library JARs, I have switched completely to tools.build
-- take a look in projects like next.jdbc
or honeysql
to see the sort of stuff I'm doing with tools.build
nowadays.
Even depstar
's own CI pipeline uses tools.build
to run tests and build its own JAR! 🙂
For most folks, using tools.build
for uberjars is going to work but there are currently some gaps between uber
in tools.build
and what depstar
does so right now I still recommend depstar
for building uberjars.
That is very helpful thank you - I will keep watching progress! Thank you.
However, I'm actively talking with Alex about this and we've been discussing a path forward to help get uber
to parity with depstar
's uberjar
functionality -- mostly around how the latter handles OSS license files in libraries that are pulled into the uberjar and also the thorny problem of log4j2 plugins cache files. My goal is to work with Alex to get tools.build
to a point where I can retire depstar
completely and recommend everyone use tools.build
instead.
(also, there's a #tools-build channel for anyone who wants to dig deeper into that stuff)
The version of Clojure CLI can be specified with the Heroku Config Car CLOJURE_CLI_VERSION
https://practical.li/clojure-web-services/projects/banking-on-clojure/deployment-via-ci.html#heroku-pipeline-configuration