Here are they again but with higher resolution and as separate pictures.

wow! how much time does poly test :project usually take? for instance if components starting with 'billing-x' is updated, lots of project will run their project tests + tests of bricks that they have.

I think @U04V70XH6 can answer that.

@U01TFN2113P if a widely-used low-level component is changed, causing everything to be retested for every project, I think the tests take about 45 minutes. That's pretty rare though since the system is, overall, pretty stable (it happens mostly when we update a widely-used library version). Prior to switching to Polylith, we ran all the tests every time, and we actually ran two CI jobs in parallel: the billing tests and "everything else". We also often needed a 2x instance for the "everything else" tests. Now we can use a standard (1x) instance for the entire test suite and it mostly runs to completion in just a few minutes for most changes. I've talked a bit about this in my various blog posts.

Polylith also lets us build only the artifacts that depend on updated code, instead of having to build all artifacts. Since we have about 20 projects, and we AOT compile for each artifact, that can be a big saving on a deployment CI job too.

I see. So since widely used components are unlikely to change, the all encompassing tests are not run so frequently.

Right. In general, we find bases - application-specific code - changes a lot more frequently than (reusable) components.

> we find bases - application-specific code - changes a lot more frequently than (reusable) components. Does this mean that the bases contain business logic codes whereas components have reusable utility codes?

My assumption is that business codes are more malleable, hence being more susceptible to changes.

The more a piece of code becomes reusable, the more stable it must be pretty much by definition. And we also have the Clojure "creed" that we try to only accrete functionally and fix bugs - we don't break things.

Bases can be very flexible since nothing else depends on them. Components must be more stable.

At work, where we've needed to change internal APIs in breaking ways, we've introduced a new name (often just adding 2 to the current name), and slowly switched code over until nothing uses the old name - and then drop the 2.

The bases contain business logic specific to a single application. We do have some business logic that is shared between multiple projects/bases but even that tends to be shared between only a few bases. For example, what happens when a member logs in -- that can happen in basically three places: the standalone login server (part of our oauth2 workflow), at registration, and a handful of auto-login (via single use token) places. The latter two are part of the main API server base. So that's two bases, two projects. So if we change the "at login" business logic, two projects need to be tested, and in those two projects, only the bases' code and the common member-login component code needs to be tested. That actually happened today and on the merge to our main branch, CI took 16 minutes to: a) regenerate API documentation (includes fetch, starting, and running and Docker image, plus uploading the generated docs -- 4 minutes), b) fetch various Docker images, c) start three Docker services (Elastic Search, MySQL, Redis), d) completely populate all three from scratch (includes 850 SQL migrations -- 1 minute 40 seconds for all of that), e) run poly test (3 minutes, 9 seconds), f) tag the release, build the API and Login server artifacts, and push them up to our staging cluster (2 minutes, 34 seconds), g) update and upload the caches (2 minutes, 45 seconds) Our staging cluster auto-deploys on a rotating 10 minute schedule so commit to availability on staging was about 25 minutes for that change. Artifacts can be deployed from staging to our entire production cluster in about 15-20 minutes (again, automated, and the business folks can do that whenever they want although they usually ask us to actually "press the button". I triggered a complete retest and rebuild of everything yesterday (by updating next.jdbc and our New Relic library which are used almost everywhere between them), and the poly test section of that CI run took 46 minutes, 40 seconds (so my "guess" of 45 minutes earlier was pretty close). That entire CI run -- which rebuilt and deployed all 20-ish projects -- took 71 minutes, 21 seconds. As noted above, step f) can take a while -- 14 minutes, 39 seconds in that case.

@U01TFN2113P Happy to answer any Qs about our setup -- via DM or in threads here.

how does > The bases contain business logic specific to a single application. go along with > A base has a "thin" implementation which delegates to components where the business logic is implemented. stated in https://polylith.gitbook.io/polylith/architecture/2.2.-base#:~:text=A%20base%20has%20a%20%22thin%22%20implementation%20which%20delegates%20to%20components%20where%20the%20business%20logic%20is%20implemented.? I originally thought as you wrote. Bases contain business logic, and components reusable codes. But upon reading the document, bases shouldn't contain (or are not encouraged at least) business logic codes.

Another question: how often do you tag your commit? I don't usually do it at work, except for when there is a release. So I opted using since: with hash for base ref to retrieve a realistic diff for the working branch.

We migrated our monorepo to Polylith so we followed the migration guide -- which says to move each application to a base. In our old monorepo, we had a subproject for each application and subprojects for the reusable code shared between them. So in Polylith we have lots of application code in each base and the components are all our reusable code. Our refactoring is ongoing -- we're slowly breaking the bases into components as well, so one day we may have thin bases and lots of components. Right now we have:

21 bases 442 files 76818 total loc,
   143 components 705 files 59400 total loc,
   21 projects 11 files 470 total loc,
  total 1158 files 136688 total loc.

We have a test-stable task in our build.clj file that runs poly test (since stable by default) and then tags our repo with stable-<username>. Each CI build on our main branch tags the repo prior to building the (changed) artifacts and pushing those to staging. That's our release tag. So we can run poly test since:release, poly test since:previous-release, etc.

The build-uberjars task in our build.clj runs poly ws get:changes:changed-or-affected-projects since:previous-release skip:dev color-mode:none to get the list of artifacts to build -- since it is run after tagging the repo. CI otherwise runs poly test since:release (prior to tagging the repo). So we only tag the repo if the tests pass (in both cases: stable locally and release in CI).

As far as we're concerned, every merge to the main branch (from a PR) that passes the whole test suite is a release candidate and can be deployed directly to production as soon as the business team feel like it.

We typically only push builds to production about every two weeks, but we have the capability to push five or six complete builds to production every day -- or over a dozen mini-builds. And we have, sometimes, pushed three or four builds of a particular project to production in a single work day.

> Each CI build on our main branch tags the repo prior to building the (changed) artifacts and pushing those to staging I don't know the specifics of your company's workflow, but if you make pull requests toward staging, shouldn't staging branch done with tests be tagged to be stable instead of main? otherwise the gap between your working branch and the release branch will get bigger over time.

While we're working locally, we might run test-stable multiple times working on a single PR. stable just says "all the tests passed for <this developer> at this point". CI tests everything changed since the last release -- which is everything changed since the last deployment to staging. Nothing is built after that. What's on staging is what gets deployed to production -- unchanged artifacts. So "the gap between your working branch and the release branch" tends to be a single PR.

(and we generally try to keep each PR to a day or less, so we work in small chunks, with the goal that every PR leads to something we could deploy to production -- feature flags help a lot here)

I tend to run test-stable after each commit -- so I'm only testing locally what changed in each commit -- and then I run test-stable again after I pull changes (whenever I go back to the main branch to start a new piece of work).

I see how it works when working locally; but how about when those commits are pushed to the remote repository? wouldn't the result of diff be large?

It's probably worth mentioning that I tend to eval each change as I make it (in VS Code/Calva/REPL) and run tests regularly -- since I have hot keys to run a) the test ns for the current source ns, b) the current test ns, and c) just the current test that my cursor is in. So by the time I run test-stable, I'm already pretty confident it will pass. We push commits in a PR. Each PR typically represents a day's work, often less, and only occasionally more. Each PR runs through CI and tends to get merged almost as soon as the tests pass -- because each PR is small and easy to review -- and so the main branch is updated multiple times a day, typically, and therefore is tagged as a release multiple times a day.

So, no, the result of the diff is small, intentionally. PRs should be small, atomic, and easy to review.

Example: we just implemented a feature to track device/browser fingerprints and allow us to ban devices (to deal with scammers trying to create fake profiles). That Epic was broken down into eight Stories -- one frontend and seven backend that included the SQL migrations, the admin app changes to review/ban/unban devices, the device tracking at registration, and the auto-suspension of members using a banned device. Each of those eight stories produced a single PR. All but one of them took less than a day to implement. The main branch was still releasable to production after each PR was merged. The entire feature took eleven elapsed days, once specification and design were done. The diffs for each release were pretty small.

That's pretty typical of our workflow -- because I'm a big believer in small PRs 🙂

Have you observed any increase in productivity since migrating to Polylith, e.g. compared to the situation two years ago?

I think the structure of Polylith has helped us focus, when writing new code -- it's easier to find when existing code might help us and the interface/impl separation makes us think harder about what code we add where -- which means that we tend to write more maintainable code, which makes it easier to enhance over time. That's a bit subjective but our velocity is generally pretty high so it's hard to measure exactly how much Polylith has affected that. In terms of CI and deployment, Polylith has been a massive improvement since our CI pipeline is smaller/faster and our deployments are smaller too. Previously, we had two parallel CI builds are it always took about 30-40 minutes for every build, even on a branch. So getting a single application change to production took at least an hour to get to staging and often an hour and a half to two hours. Now those changes can get to staging in half the time or less. We get stuff into the hands of the business team faster.