We've been working on replacing our build shell script with a Clojure build.clj script and we got part-way there and we're "happier". We've gone from several hundred lines of bash down to 100 and build.clj is about 250 lines of Clojure. For "build"-style stuff, it's all good, but we'd like to be able to integrate test setup and execution (DB migrations, running tests) into a single pipeline with build stuff (tagging, pom.xml sync'ing, AOT, JAR creation, deployment). That would mean running tests in the "context" of t.d.a which our build.clj relies on for the "build" stuff -- and t.d.a is a big ol' bag of dependencies that might interfere with running our code. So we find ourselves either running tests separately with just the "dev" dependencies -- and therefore requiring a "driver" script fronting the CLI (to do setup and run tests, and then to run our build.clj script for the rest of it) -- or running tests in the context of dev+t.d.a which doesn't feel ideal (since t.d.a doesn't end up in our artifacts). And that means we find ourselves wanting something like "Boot pods" to provide classpath isolation within a single JVM to run tests with a fixed set of dependencies...

(it's interesting to see that Polylith's poly tool has actually gone down this path for running tests so that it can isolate dependencies on a per "project" p.o.v)

Given the imminence of tools.build (he said, hopefully), what do people think about tooling that integrates test running with artifact building (to perhaps avoid spinning up multiple JVMs)? Is this a worthwhile goal or is it just not worth the hassle?

(Both Boot and Leiningen support this model of issuing a single command to run tests and build artifacts but I don't know how widely that capability is actually used in practice?)

@seancorfield why do you need t.d.a on the classpath after it's done resolving deps / calculating a classpath?

Because build.clj uses t.d.a to calculate project basis information for various tasks.

e.g.,

(with-dir (io/file (str (System/getProperty "user.dir") "/projects/" p))
          (let [{:keys [success] :as result}
                (uberjar (-> (calc-project-basis) :aliases :uberjar :exec-args
                             ;; cater for being _above_ the project dir:
                             (update :jar #(str "projects/" p "/" %))))]
            (when-not success
              (throw (ex-info (str "uberjar failed for " p) result)))))

can you compute the classpath and then set that manually using -Scp for the real tests?

(`with-dir` is from clojure.tools.deps.alpha.util.dir)

The point is to avoid spinning up additional processes/JVMs here.

That fragment above is run for each (sub) project that we want to build a JAR for.

Building a jar for a project doesn't actually need deps of that project on the classpath right?

You're missing the point.

I already said that for build stuff, it all works as we want. Running tests is the problematic piece.

gotcha

I just used the above as an example of why/how we're using t.d.a in our build script -- as tools.build is going to do too, I'm sure.

So you want different dependency sets for each test suite, within one JVM?

Ideally, yes.

Which is what Boot could do, and what Polylith's test runner does -- and once we're 100% Polylith, we can just rely on that 🙂 but right now we're a mix of legacy subprojects and Polylith "bricks".

I think I would just run multiple JVMs as the other options sound a bit brittle, but this may just be my inexperience with the boot pod stuff.

It's fine if the expectation is "Run tests standalone, run build stuff together" and the ability to run tests in isolation is just a non-goal for tools.build but given Leiningen/Boot's ability to "run tests and build stuff" all-in-one it seems reasonable to ask 🙂

It's true that Boot's "pods" were indeed problematic but mostly around the supposed optimization of async refreshing and reusing of contexts.

I've built quite a bit of "basis-making" stuff in the imminent release. I have not done the work to potentially use it to feed something like test-runner but theoretically, it's a small gap

The "gap" seems to be around classpath-isolation, unless the tests are run in a separate process (which was what we were trying to avoid).

I'll take a look at that today - it has a couple possible interaction points of interest. tools.build has a facility for spinning processes and some stuff spinning from basis inside compile-clj but the useful bits are not abstracted out and that's been on my list to look at for a while

are you trying to avoid it as "having 2 things" or avoiding it because it's slower?

For now, it's not really a huge deal for us: we can run the tests in the same context as t.d.a (and we actually have one test that is connected to our build stuff that requires t.d.a) but t.d.a has such a huge spread of transitive dependencies. I think @hiredman figured out there are about two dozen deps that overlap with our code's deps that have different versions.

Mainly b/c we'd like to have a single clojure -X:some:aliases do a bunch of things rather than finding ourselves wanting a bash script to front that, just so it can determine that we really need to run clojure -X:some do a bunch and then clojure -X:some:aliases of things 🙂

believe me, I get it re deps :)

yeah, some of the new basis stuff would help you here. the cool thing about the basis is it's a Clojure map so perfectly amenable to being made in one jvm (and/or cached) and picked up for use in another

Or even passed back and forth between two classpath-isolated "processes" in the same JVM 🙂

yes, although I'm not rushing to integrate that directly :)

I sometimes ponder whether you could have an architecture where you ran a "basis server" that just computed (and cached) classpaths, and another that spun empty jvms, then dynamically updated their classpaths loaded from the basis server

maybe that's reinventing nailgun, I dunno

A more narrowly-focused problem to solve is just a test runner that is driven by deps.edn for a monorepo -- where different subprojects can have different deps. It's exactly what poly attempts to do so we may well just punt on this as something worth even solving since it will eventually "go away" for us. And it may well be that this is just a non-problem for folks who have a single testing context (i.e., not a monorepo).

(the poly test runner also has the benefit of incremental testing -- so it only runs tests that could be affected by files that have changed, and optionally for a specific "project" p.o.v)

I'm working (still exploring the possibilities) on an experimental native version of tools.deps.alpha. I don't expect Sean to adopt this (given its non-official status), but just for the sake of argument: it can calculate classpaths (or basis-es) very very fast, without spinning up a JVM. This would cut out one JVM instance at least, for the scripting part. But you would still have the "I want one JVM to run with different dep combinations/classloaders" problem, which is probably the biggest challenge anyway.

There is a working version to download here: https://github.com/borkdude/tools-deps-native-experiment

"I don't expect Sean to adopt this (given its non-official status)" -- you know me well @borkdude 🙂