Fork me on GitHub
#tools-deps
<
2021-03-26
>
rickmoynihan10:03:26

I just ran into a small issue with org.clojure/tools.namespace, where when used in a deps.edn project would over-zealously by default load namespaces found in ~/.gitlibs. Presumably because prior to tools.deps it was assumed deps found on the filesystem were most likely to be source code in your project and suitable for loading. Would it make sense for tns to now by default ignore any paths found in ~/.gitlibs?

Alex Miller (Clojure team)10:03:00

I don’t understand - can you explain a repro in an ask.clojure question?

rickmoynihan10:03:45

Incidentally this is easily fixed by explicitly setting tnsrepl/set-refresh-dirs, but it did cause a colleague to get a very misleading stacktrace about a missing class from a namespace that his project wasn’t explicitly requiring.

rickmoynihan10:03:24

I should add that they likely weren’t even fully aware they were using tns, because they were using integrant.repl/reset (which uses it)

seancorfield16:03:59

Cognitect’s test-runner seems to be accumulating issues and PRs without getting any feedback from Cognitect. Is it just considered “done” or just very low priority right now I wonder? I was thinking it would be really nice to have a -X entry point so that folks could leverage :exec-args and, in particular, their ability to merge across aliases so you could specify generic options via your :test alias (or :runner alias, in my case) and then specific options in other aliases. This would clean up a bit of boilerplate in our monorepo scenario where subprojects all have aliases but we still need to specify --dir via the command-line (since :main-opts don’t merge — just last-one-wins).

seancorfield16:03:18

Luckily the test function in cognitect.test-runner is public so writing a tiny :exec-fn wrapper outside the runner is easy to do for now.

Alex Miller (Clojure team)17:03:30

Just waiting for a cycle of attention

seancorfield17:03:23

I’ll write the wrapper in our codebase for now and maybe submit a PR if/when test-runner gets an attention cycle.

borkdude18:03:20

The test runner is working well for me

borkdude18:03:24

@seancorfield I'd love to see an example invocation that is "better" with -X than with the -main function

seancorfield18:03:55

We have a test alias for each subproject that brings in the test code and dependencies for that subproject. Via -X we can specify :exec-args {:dir #{"subproject/test"}} which will merge with our :runner alias which already has :exec-fn and the default :exec-args.

seancorfield18:03:43

With -M, :main-opts do not merge so we either have to repeat the whole :main-opts for every subproject -test alias with just -d changed, or we have to specify -d on the command-line.

borkdude18:03:22

so now you have to write another alias for every -d change?

seancorfield18:03:31

Before:

$ clojure -M:defaults:subproj:subproj-test:test:runner -d subproj/test
After:
$ clojure -X:defaults:subproj:subproj-test:test:runner

borkdude18:03:55

@seancorfield Do you store these long invocations in some separate bash script or Makefile? Or do you remember them from the top of your head?

borkdude18:03:24

I'm currently considering a task runner which is supposed to solve this problem (of not having to remember these things)

borkdude18:03:48

This is why I am curious about these use cases

seancorfield18:03:59

We have a build shell script that turns build test subproj into the above command. But now that we have a primary deps.edn file at the top of our monorepo, it’s easy enough to run most things with the CLI directly (for single command invocation).

seancorfield18:03:39

The build script supports multiple commands so we could say build tests subproj1 subproj2 and it runs two clojure commands.

borkdude18:03:05

The example after -X is still pretty long, too long for me to remember without a bash history probably

hiredman19:03:04

Why would you ever not have a history?

borkdude19:03:17

That can happen when I switch systems yes, I have multiple laptops. But even then, fetching from history comes up with similar invocations and not always the one I'm looking for right away, unless I remember it more or less correctly. Hence, something like a task runner (something like make) could be nice

borkdude19:03:11

But also for making these invocations known to colleagues, contributors, etc, you don't want to depend on history

hiredman20:03:19

maybe; I am skeptical of lot of this additional tooling stuff. shells already support most of these things

hiredman20:03:39

everything goes in the history

hiredman20:03:57

support running all kinds of things, background jobs, etc

borkdude20:03:27

you don't write any docs for your colleagues how to invoke the tests like Sean showed? I would have a hard time figuring that out by only looking at deps.edn

hiredman20:03:30

it is already the case that I almost never type a command in full into my shell, I pull it out of history and maybe edit it

hiredman20:03:55

if you write docs then you already may as well just share the long command

hiredman20:03:17

"paste this into your shell, and it will run the tests"

borkdude20:03:20

if you do document this, you might as well make those docs executable somehow

borkdude20:03:38

yeah, that's usually how I've done it so far

hiredman20:03:40

I mean, I work with sean

borkdude20:03:08

Lucky you ;)

hiredman20:03:09

yeah, I mean, he does a lot of work on the tooling, so documentation wise it is way more likely to be written for me then by me :)

hiredman23:03:19

This makefile is bad

hiredman23:03:46

Like, the whole deal about makefiles is it tracks dependencies between files and has tasks that generate files

hiredman23:03:51

This make file has a target named 'pom' that is used to generate the file pom.xml

hiredman23:03:21

Which has no dependencies (like maybe on deps.edn?)

hiredman23:03:28

This task dsl is gross too. To impose a restrictive dsl without getting anything out of it (like dependency tracking) is silly. If you aren't doing extra stuff (dependency tracking, staleness tracking, rebuilding, etc) the just use normal clojure functions

hiredman23:03:19

You want give some bit of functionality a name, and call it, we have defn for that

borkdude23:03:44

This isn't my makefile btw, I just scouted some makefiles in the wild, and what I mostly see is that people just use it as a way to quickly invoke it from the command line. It's clear that you don't see the value of this, that was already clear in our earlier conversation. Thanks for the feedback.

borkdude23:03:20

I do want to consider tracking, etc, like make does, it's done done yet

borkdude10:03:21

Now ported the tasks to normal functions: https://github.com/borkdude/antq/blob/bb.edn/script/tasks.clj The code longer, but more flexible, so that may the way to go indeed. Tasks can be discovered using (maybe warrants a bb dir CLI option) :

$ bb -e "(require 'tasks) (clojure.repl/dir tasks)"
clean
coverage
deploy
docker
docker-test
install
jar
jar-file
lint
outdated
pom
repl
standalone-jar-file
tests
uberjar
Tasks can be invoked using:
bb -m runner/coverage
Docs:
$ bb doc tasks/coverage
-------------------------
tasks/coverage
([])
  Run test coverage.

seancorfield18:03:24

We’re not going to invest more work in pipelining this stuff until we’ve seen tools.build released.

seancorfield18:03:29

I’m anticipating being able to replace our build shell script with a tools.build invocation 🤞:skin-tone-2: 😸

borkdude18:03:28

I don't know what tools.build is, nor spec2 ;)

borkdude18:03:01

But I hope they come out soon ;) 🔮

seancorfield18:03:17

Based on what Alex has said publicly about tools.build, I would expect it to include “task runner” functionality (and some “standard” tasks). ISTR he’s already mentioned that it will add Java compilation as a task?