Fork me on GitHub
#tools-deps
<
2021-06-26
>
seancorfield06:06:56

How stable is this part of the tools.deps API:

user=> (require '[clojure.tools.deps.alpha.util.dir :refer [with-dir]] '[hf.depstar :refer [uberjar]] '[ :as io])
nil
user=> (with-dir (io/file (str (System/getProperty "user.dir") "/projects/api"))
         (uberjar {:jar "../../../build/uberjars/api-2.0.0.jar" :main-class 'api.main :aot true}))
[main] INFO hf.depstar.uberjar - Compiling api.main ...
[main] INFO hf.depstar.uberjar - Building uber jar: ../../../build/uberjars/api-2.0.0.jar
nil
user=> 
We're going to start replacing our build shell script stuff with a Clojure build.clj script or something similar and the shell script cd's into subproject folders to run commands and I figured out the above is equivalent... but it feels a bit like I'm relying on some internals there? I'm hoping with tools.build that there's some standard way to specify the "user directory" for running different tasks so a build script can navigate around a repository and run tasks in different places?

dominicm08:06:32

I assume the aim is to avoid multiple jvms, so a faster build overall?

seancorfield08:06:19

Yes, partly. And also just to get rid of bash scripts where possible šŸ™‚

seancorfield08:06:46

Given that tools.build is going to be taking us in this sort of direction, it seems reasonable to start planning.

seancorfield08:06:58

I just posted in #polylith a function I created that builds an uberjar for every projects entry in a Polylith workspace, which is where we're headed with this.

seancorfield08:06:26

(but it's applicable anywhere that you have multiple subprojects that need a task applying to them in sequence)

seancorfield08:06:23

My next task is to figure out whether classpath-isolated subproject test runners are achievable šŸ™‚

seancorfield08:06:28

(given that Polylith already has a version of this in its command-line tool, the answer is obviously "yes" but I just need to figure out how much work that is...)

dominicm08:06:29

Only polylith projects have that issue of needing classpath isolation because of how it swaps out namespaces right? Otherwise you could just load them all into a single JVM?

dominicm08:06:30

(If clojure.test could be parameterized, you wouldn't even need to do that, you could just use a protocol!)

dominicm08:06:36

@U04V70XH6 for cp isolation, you might be interested in https://github.com/SevereOverfl0w/clj-embed which was forked to fix a few bugs I found in upstream. It's behind the API times, but it's probably faster than shelling out.

seancorfield16:06:30

Thanks @U09LZR36F and, no, re: Polylith swapping namespaces: this is more about wanting potential classpath isolation between subproject test runs. We have one particular subproject that requires a specific, old version of Jackson (before they changed the null merging behavior in 2.9.0) but the other subprojects can run with "any" version of Jackson. I haven't decided yet how I want to approach this part. Currently, we run a separate clojure process for each subproject when running tests but that's (obviously) slower than just loading everything and running all the tests in a single JVM...

seancorfield16:06:37

Looks like clj-embed does "more" in terms of isolation than I'd really need but I suspect it would get me closer. Thanks.

athomasoriginal18:06:09

Hi all. Iā€™m just getting around to tackling the https://clojure.org/releases/tools#v1.10.2.790 https://clojure.org/releases/tools#v1.10.2.790 warning now for my monorepo. I have read through Seanā€™s posts and conversations in this channel about monorepo setup with this new constraint. I have outlined my https://gist.github.com/athomasoriginal/47913c6b946fc7416c3d839be101e600 and wanted to see if this aligns to the ā€œrecommended approachā€? > I use ā€œrecommendā€ lightly. My goal is to be sure the approach is not going against the grain too much or setting myself up for future pain. Thanks!!

seancorfield18:06:40

Sounds about right but I'll elaborate a bit on a couple of our monodeps setup...

Alex Miller (Clojure team)18:06:09

I assume you mean :extra-paths in there, not :external-paths

šŸ‘ 3
seancorfield18:06:18

Our "workspace-level" deps.edn has an :everything alias which has :extra-deps listing each subproject as a :local/root dep and :extra-paths listing each subproject's test folder -- we use this alias for starting a global REPL to work with the entire codebase. I think that's an important affordance.

seancorfield18:06:03

That top-level deps.edn file is where we declare any/all tooling -- but we don't have cljs subprojects so I don't know how that would change the equation (if at all).

Alex Miller (Clojure team)18:06:47

why are app and app-server separate? app is what you build and app-server is what you deploy?

Alex Miller (Clojure team)18:06:03

one thing coming is the ability for projects to publish that they must be "prepared" before use and for consumers of that project to ask to do that preparation

Alex Miller (Clojure team)18:06:01

I'm trying to understand your bigger use case here and whether that could be used for advantage (having app-server consume app as a local dep, which has a prep step)

athomasoriginal18:06:34

> I assume you mean :extra-paths in there, not :external-paths Yes. My bad šŸ˜ž

athomasoriginal18:06:42

> why are app and app-server separate? ā€¢ app is the ClojureScript application (client) ā€¢ app-server is the web-server serving static assets and a REST API My preference is separation because iā€™ve found it makes it easier to understand the backend/front end divide and helps avoid coupling tooling. Before I go further though, does that clarify? Iā€™m not sure I understand your comment: ā€œapp is what you build and app-server is what you deployā€ :thinking_face:

seancorfield18:06:19

I suspect app-client and app-server might have been less confusing names šŸ™‚

athomasoriginal18:06:49

I think you are correct šŸ˜‰

Alex Miller (Clojure team)18:06:14

ah ok, that makes sense

šŸ™ 3
athomasoriginal18:06:06

I think I understand your build v. deploy question now though šŸ™‚

athomasoriginal18:06:02

> Iā€™m trying to understand your bigger use case here and whether that could be used for advantage Does the use case make more sense now? It might be outside the scope of the feature youā€™re considering though?

Alex Miller (Clojure team)18:06:20

not sure, might still be a good match?

Alex Miller (Clojure team)18:06:30

app would have some build process and publish in its deps.edn how to run it. use :path including the output of that process. app-server would depend on app

athomasoriginal18:06:59

Yeah, thatā€™s it.

Alex Miller (Clojure team)18:06:00

there may be some tooling that will need to catch up to all that though

athomasoriginal19:06:02

> That top-levelĀ `deps.edn`Ā file is where we declare any/all tooling @U04V70XH6 I am re-reading your post again, but in my scenario my driver for relying on paths external to the app-server project was only so I could have access to app-client s resources and target dirs because I feel itā€™s awkward to have those specified in the app-server project while your developing the appā€™s front end. So I believe a difference is that I donā€™t actually need to work with clj code, I just did it to serve static assets in development.