Fork me on GitHub
#tools-deps
<
2022-05-24
>
Sam Ritchie16:05:42

hey all — I am looking to run clj -P in my github action, but I want to prepare the deps for one of my aliases. Following this advice: https://gist.github.com/realgenekim/97cd8da28b00761560bef06b8072eb9f I ran clj -A:test -P which works! but gives me this warning:

WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
is there a non-warning way to write this?

Sam Ritchie16:05:59

[sritchie@wintermute ~/code/clj/sicmutils (main)]$ clj -M:test -P
Unknown option: "-P"
it’s not that

Alex Miller (Clojure team)16:05:28

-M and -X capture everything after as args

seancorfield17:05:06

@U017QJZ9M7W And remember to add in any other aliases you might need for build tooling etc so those don't get downloaded every time (I assume you're doing this to set up caching?). Our CI pipeline has:

clojure -P -M:build:poly:everything:dev:test:runner

Sam Ritchie17:05:46

I have separate github actions steps now for lint, test and coverage, so I was doing only the ones I needed to cache for each step, but that’s an excellent tip. @U04V70XH6 I suspect I’ll have more questions… I’m doing a full switch from leiningen now to get ready for modularizing #sicmutils. next step is to get a clojars publish step going,

seancorfield18:05:56

@U017QJZ9M7W Sure. Feel free to DM me if you have Qs (or ask them publicly).

❤️ 1
Sam Ritchie18:05:01

@U04V70XH6 actually here is one. is there a way to stack aliases? for example I am setting up cloverage now and realizing that I need all test deps and namespaces (of course). I’m asking because I THINK i remember seeing that this is not a thing…

Sam Ritchie18:05:29

trying this clj -M:test:coverage --codecov

Sam Ritchie18:05:40

that seems to work; but semantically seems wrong since coverage won’t work without test , ie folks will definitely forget this

seancorfield18:05:47

Currently, there's no way to have an "alias of aliases" but it is a common ask -- and there's at least one http://ask.clojure.org Q about that and/or similar things.

seancorfield18:05:00

We have split our :test alias and our :runner alias so that :test is easier to combine with other aliases -- and we have :coverage as an alias, in fact 🙂 Also, since we are migrating to Polylith, we have a :dev alias which is "our production Polylith code" and :everything which is "our legacy production code" so clojure -M:dev:everything is kind of our baseline for any tooling to make any sense of our code...

seancorfield18:05:35

FWIW, here's how I start my development REPL as an example of combining aliases:

SOCKET_REPL_PORT=none clojure -J-Dlog4j2.configurationFile=log4j2-sean.properties -M:build:dev:test:everything:runner:add-libs:jedi-time:nrepl:portal:reflect:dev/repl

Sam Ritchie18:05:48

awesome, I found your project here too, this is a huge help https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn

seancorfield14:05:39

@U017QJZ9M7W just a caution: although that's up on GH and I use it as an example for what can be done with deps.edn, it's my personal config and subject to change without notice -- and using "RELEASE" for tools is considered poor practice (and that symbolic version is deprecated by Maven).

Sam Ritchie14:05:47

It's a great cookbook and really helpful to see patterns. The linter has already scolded me about RELEASE and I am a version pinner anyway so hear you there too

📍 1
seancorfield14:05:37

Everything inside my projects is always pinned. And then I run antq periodically to let me know about potential updates (it's not in an alias -- it's via a bash script that fancies up the report quite a bit).

Sam Ritchie16:05:33

That's great, I am going to need some custom goodies to get these multiple modules published, but locally linked for development.

Sam Ritchie18:05:21

is there a pattern for starting a repl a namespace other than user? This seems not to work, I stay in user:

clj -M:test:dev/repl -r -e "(require 'sicmutils.env) (in-ns 'sicmutils.env)"

Drew Verlee23:05:40

This question confuses me a bit. I tend to start a nRepl server and client (connected to my editor) and i send expressions e.g (inc 1) from the client to the nRepl server which reads, evals, then prints it. Well it doesn't print it really, it makes message and sends it back to the client so it can print it. what does the alias :dev/repl do? nRepl server is used by a couple editor programs like cider w/emacs. But the idea is the same for all the editor's (vscode, intelliji, vim, what-ever, etc...)

Drew Verlee23:05:09

What i'm saying is, it looks like your trying to eval things from the command line using -e, but you would be better served using your editor to eval expressions.

Sam Ritchie23:05:22

I want to provide an experience for new users that will allow them to write a command that drops them into a REPL with all batteries included, but configured so they understand what namespace they’re in , ie, where all those batteries live

Drew Verlee23:05:28

If your going to use the command line, your going to have to print things then right? Like you could print a message when they run that command that introduces things.

Sam Ritchie23:05:11

Yeah that's a good idea too!

Sam Ritchie23:05:06

I had wanted to avoid simply pulling everything into the user namespace as it feels too magical, and it doesn't give someone a hint of “ah, I can go read this source file or cljdoc page to see everything configured”

Sam Ritchie23:05:48

Thanks!! That's where I'm doing these tools.deps shenanigans https://github.com/sicmutils/sicmutils/pull/517

Sam Ritchie23:05:26

@U0DJ4T5U1 the project is this huge foundation… the next phase is a bunch of visualizations and a pair of executable textbooks built on this code. I'm building up a new head of steam to get that work out, and I hope it will be more broadly compelling (think Mathematica-lite that can run on server or browser etc)

Sam Ritchie23:05:52

@U0DJ4T5U1 I'm in new dad mode with a pair of twins so that comment gave me a much needed smile :)

Drew Verlee00:05:09

no worries. The other day i was remarking that we (humanity/me) need a way to bridge math symbols and programming.

Sam Ritchie18:05:57

ah! putting -r at the END works.

Sam Ritchie18:05:56

clj -M:test:dev/repl -e "(do (require 'sicmutils.env) (in-ns 'sicmutils.env) (str \"Clojure \" (clojure-version)))" -r
this gives an experience ALMOST like bare clj

Sam Ritchie19:05:13

for publishing to clojars; is a good pattern here to keep a pom.xml checked into the repo, and modify the deps with clj -Spom?

Sam Ritchie19:05:40

I like this, but this feels like this is going to be tough when I go to modularize the build (because each subproject will have its own pom file etc…) so maybe I will do this for now, pause and then figure out how to go modular.

seancorfield20:05:11

I keep a "template" pom.xml checked into git and generate the real one as part of the build process for deployment. See honeysql or next.jdbc for examples.

seancorfield20:05:21

(I don't run clojure -Spom -- I do everything via tools.build because you have more control)

Sam Ritchie15:06:42

I think I’ve got this all set up now

1
Sam Ritchie15:06:29

the only improvement would be the ability to consolidate everything into build.clj, so no one has to run combos of -X , -M with stacks of aliases etc

Sam Ritchie15:06:45

@U04V70XH6 it looks like your run-task is the best model for that, yeah?

seancorfield15:06:09

That lets you run any -M style task in a subprocess.

seancorfield15:06:47

(not every task needs to be run in a subprocess -- it does add some overhead -- but it provides isolation)

Sam Ritchie15:06:56

I’m a little surprised tools.build is missing this (interaction between -T tasks and clojure’s -M and -X runners, etc

Sam Ritchie15:06:23

is there some built-in way to trigger, say, clojure -M:eastwood from clojure -T:build without run-task?

Sam Ritchie15:06:50

the only goal being to replace clojure -M:test:coverage and these various tasks that require stacked aliases with, say, clojure -T:build coverage so I can bake in the stacking

seancorfield16:06:36

We run tests and cloverage via run-task as separate subprocesses so that we get isolation from the tools.build/tools.deps dependencies. Alex has said that it's sort of expected for folks to just run those directly via the CLI and not use build.clj for it, but I think it's "expected" by users that clojure -T:build ci should run the CI pipeline precisely so you don't have to run multiple CLI commands directly.

seancorfield16:06:15

For a while -X let you execute multiple functions in a pipeline but that was removed (there were some unresolvable conflicts in how to parse functions, symbols, and command-line arguments) so "task pipelines" were supported directly in the CLI. I think task composition is a natural thing to want, and isolation is also a natural thing to want for some tasks...

seancorfield16:06:54

At work, we have a way to run -X/`-T` tasks in a subprocess but it relies on some pretty awful hacks so I haven't released it.

Sam Ritchie19:06:22

I can see now how I’ll do the multimodule setup. Probably hardcoded local dependencies in the git repo between modules, and then I’ll override these in my build script on a local install or remote publish

Sam Ritchie19:06:16

it is nice to have simple pieces and the full language available. At least, I am deep enough into Clojure, build tooling etc that this feels great and non-confusing. There’s a certain level of confusion in the problem domain that’s hard to get past for someone just beginning, I’d imagine

seancorfield19:06:58

Yeah, I think tools.build is the first time we've all been really exposed to the concepts behind the CLI in terms of computing a "basis" from one or more deps.edn files and using that to run other Clojure tasks... Folks who've built CLI tools using t.d.a. have dealt with this to some extent before, but now it's very much a public API for writing build systems.

👍 1