Fork me on GitHub
#polylith
<
2021-12-01
>
vemv03:12:43

Kind of a silly question because I mostly know the answer / the phenomenon is not unique to Polylith. But: isn't it sort of a bummer having to restart repls every time one adds a new source path? This can make it tempting to make things less fine-grained as they ought to be for an ideal design. e.g. poly create c name:my-component creates a new source path, which didn't exist in the classpath in the already active repl, so one would have to restart the repl for having the new component integrated with a variety of tools (from the top of my head: tools.namespace, cider-nrepl, eastwood, refactor-nrepl). Probably kondo+lsp as well, they also need a comprehensive classpath for accuracy. ...One can eval forms in the repl though so one technically can make the namespaces exist at least / test may pass as well. Curious if this is a well-discussed pain? Not that it would be impossible to solve anyway

seancorfield04:12:27

I don't restart my REPL. When I add a new brick, I just "load file" the new interface and impl files and then they're in memory. LSP/clj-kondo seems to cope with this pretty well. I'm using VS Code/Clover (with Calva/LSP for the static stuff).

đź‘€ 1
seancorfield04:12:52

My REPLs stay running for weeks, in general.

catjam 1
seancorfield04:12:36

I do not use tools.namespace or any sort of reset/refresh tooling.

seancorfield04:12:10

I've been doing some pretty heavy refactoring from our legacy subproject structure into Polylith bricks without needing to restart my REPL.

vemv04:12:38

forgot to say, thanks! (yesterday was definitely late around here) Useful info as always.

1
winsome03:12:50

Certainly would be a good quality-of-life improvement : )

Joshua Suskalo17:12:15

What's the reasoning for tools.deps being a default dependency in the dev project?

tengstrand18:12:22

Actually, there is no reason (it was added a long time ago)!. I will add it to my todo list, to remove that dependency from the “create workspace” command!

seancorfield18:12:52

I think it also gets added to create project?

seancorfield18:12:31

Yeah, it produces this deps.edn:

{:deps {org.clojure/clojure {:mvn/version "1.10.1"}
        org.clojure/tools.deps.alpha {:mvn/version "0.12.985"}}

 :aliases {:test {:extra-paths []
                  :extra-deps  {}}}}
It's somewhat questionable whether it should include a default Clojure version but it definitely should not include tools.deps.alpha.

tengstrand19:12:01

Yes, it should be removed from there too. Maybe it’s okay to add the clojure dependency, because it’s a reasonable default, and it’s easy to remove or change the version of.

Joshua Suskalo19:12:00

Thanks for the responses! Maybe the clojure version could be pulled from the workspace? I figure each project should have a clojure dependency by default (but not e.g. a base or component), but it might make sense for the default to be configurable.

seancorfield07:12:20

@U5NCUG8NR That's a really interesting idea. I would definitely like to be able to specify the Clojure version just once, somewhere, instead of having to put it in every project (esp. since I'm using Clojure 1.11 Alpha 3 for everything now).

tengstrand06:12:59

If I understand right, the version of org.clojure/tools.depsshould be configured ing workspace.edn and then be used when creating projects? Maybe a hash map with default dependencies and their versions could be the solution, because then people could decide exactly how the initial set of dependencies should look like for new projects.

seancorfield06:12:57

Not org.clojure/tools.deps.alpha -- org.clojure/clojure.

seancorfield06:12:33

t.d.a should not be a dependency in anything the poly tool creates (that's kind of a bug right now).

Joshua Suskalo19:12:48

When building projects with polylith, how do you normally go about specifying an entrypoint? For example: I am using tools.build to build an uberjar from a project, and I specify a class name that's specific to the project, which references some namespace in the base that it includes. That namespace in the base includes a :gen-class form, and so it expects to be compiled before it's included in an uberjar. Unfortunately it seems to me to be more than a little difficult to depend on a base that requires compiling a main ns because if I were to specify a way to compile that ns as a prep-lib step for the base, then it can't be prepared because the base doesn't directly depend on the components which provide the namespaces that it would have to require. I know the "recommended" way to do this in polylith is to use seancorfield's build-clj which allows you to specify aot on the uberjar task instead of directly using compile-clj from tools.build, but I was curious if anybody is doing this without depending on build-clj.

seancorfield19:12:02

Each project has its own build process and knows about all the components that go with the base.

Joshua Suskalo19:12:31

Ah, so the idea is that it's per-project builds, rather than per-workspace abstracted builds?

Joshua Suskalo19:12:45

That's what we'd been shooting for.

seancorfield19:12:48

There's no magic in build-clj - it's just calling those functions for you

Joshua Suskalo19:12:34

So that would mean that build-clj cannot aot-compile namespaces outside of the current source directories? It was my understanding that if it worked like depstar and similar tools that it could aot anything on the classpath of the result.

seancorfield19:12:56

At work, we do have a single workspace build script - I'll hook more about that when I'm back at my computer

seancorfield19:12:23

Those namespaces ARE on the classpath for the project

Joshua Suskalo19:12:20

Yes, they are. The challenge I was running into is that compile-clj needs a source directory, which means I'd have to determine what the source directory is for the base in the project. I think I can derive that from the main ns for our project though.

Joshua Suskalo19:12:27

Thanks for the talk, I think I've figured out how to proceed.

seancorfield19:12:29

It only needs :src-dirs if you tell it to compile :all namespaces.

seancorfield19:12:52

That can be empty if you specify :ns-compile

seancorfield19:12:16

And each project knows the entry point (main namespace). No source path derivation needed.

seancorfield19:12:59

Compilation is transitive so you only need the main namespace and then compile-clj will find that on the classpath and compile it and everything it calls.

seancorfield19:12:59

If you have dynamic requires, you would need to ensure you copy the source files into the JAR as well as the compiled class files (but lein uberjar does that anyway, and that's the assumption behind depstar and build-clj too).

Joshua Suskalo20:12:44

Oh, I hadn't realized that the src-dirs could be empty

seancorfield20:12:10

Here's how we handle the workspace-level abstracted build at work:

(doseq [p projects]
      (println "\nRunning: uber on" p)
      (let [project-dir (str (System/getProperty "user.dir") "/projects/" p)
            aliases     (with-dir (io/file project-dir) (get-project-aliases))]
        (binding [b/*project-root* project-dir]
          (bb/clean {})
          (b/copy-file {:src version-clj
                        :target "target/classes/ws/uberjar/release.clj"})
          (bb/uber (assoc (:uberjar aliases)
                          :compile-opts {:direct-linking true}))
          (bb/clean {}))))
and then each project has an :uberjar alias like this:
:uberjar
  {:uber-file "../../../build/uberjars/api-1.0.0.jar"
   :main api.main}
so that's how it gets the main namespace from the project deps.edn and also the location of the JAR file to create (we put them in a specific place, relative to the Polylith code).

seancorfield20:12:35

[clojure.tools.deps.alpha.util.dir :refer [with-dir]]
in case you're wondering about that function 🙂

seancorfield20:12:15

And version-clj is a file containing a def of version to a string containing the git version at build time.

seancorfield20:12:38

(let [version-clj (str "/tmp/version" (rand-int 10000) ".clj")]
    (spit version-clj (str "(ns ws.uberjar.release)"
                           "(def version \"" (git-version) "\")"))

Joshua Suskalo20:12:41

Thanks! That's really helpful.

seancorfield20:12:43

And, finally, we invoke that via the REPL as (build/uberjars {:projects '[api auth login]}) for example and via the command-line as clojure -T:build uberjars :projects '[api auth login]'

Joshua Suskalo19:12:44

(we'd been avoiding including it because it just felt a little too big, but this application may end up being a reason to include it)

seancorfield20:12:40

Wait, you think build-clj is "a little too big"?

seancorfield20:12:04

Because of the deps-deploy dependency?

Joshua Suskalo20:12:59

Yeah. This may be an odd opinion, but me and my boss are preferring just using tools-build instead of importing the defaults of build-clj for deployments, uberjarring, etc. We just don't need almost any of the tasks in it, we would be importing it just for the uber function and that's it.

Joshua Suskalo20:12:31

and the only thing that uber provides as a convenience over what we'd do ourselves is incorporating the call to compile-clj that we have to make to get our main namespaces compiled

seancorfield20:12:25

And the log4j2 plugin cache conflict handler. Very important if you use log4j2 🙂

seancorfield20:12:49

Although you could depend on that separately and specify it yourself to tools.build.api/uber

seancorfield20:12:39

At work, we don't use deps-deploy at all but we still depend on build-clj for the convenience it brings 🙂

seancorfield20:12:08

Would it change your mind if I added a new entry point to build-clj that didn't depend on deps-deploy? You could specify :deps/root "slim" in the git dep and it would not drag in deps-deploy and all its transitive deps?

Joshua Suskalo20:12:03

Ah, good to know about the log4j handler.

Joshua Suskalo20:12:31

And since this is only a build-time library, the actual dependency on deps-deploy isn't the most important part here, it's just that it's pulling in build tasks that we wouldn't use, like clean, test, etc.

Joshua Suskalo20:12:54

we don't have anything against the library, it's just that it's got lots of features and some defaults that are outside what we needed, and I/we don't like bringing in libraries to use one function mostly.

seancorfield20:12:53

These are the only dependencies:

{:deps
 {io.github.clojure/tools.build {:git/tag "v0.6.8" :git/sha "d79ae84"}
  io.github.seancorfield/build-uber-log4j2-handler {:git/tag "v0.1.0" :git/sha "ab8e499"}
  slipset/deps-deploy {:mvn/version "0.2.0"}}}
There's no dependency on test runners. So you get 300 lines of code, about 100-150 of which you won't use 🙂

seancorfield20:12:18

I'm updating it to exclude deps-deploy via the slim entry point since that would be useful for us at work anyway.

Joshua Suskalo20:12:41

Sounds good. I know now that I understand tools build I'll probably be using build-clj for some of my personal projects going forwards. I'll look at it again to see if it fits with what we need at work. I suspect if we end up needing log4j handlers we'll probably end up using it, although I see the handler is available as another dependency.

seancorfield20:12:16

OK:

:build {:deps {io.github.seancorfield/build-clj
                 {:git/tag "v0.6.0" :git/sha "2451bea"
                  ;; omits deps-deploy dependency:
                  :deps/root "slim"}}
          :ns-default build}

Joshua Suskalo20:12:53

that was fast, thanks!

seancorfield20:12:39

We're already using it at work 🙂