Clojurians Log v2

The design decision behind this was that we wanted to isolate the development dependencies from other aliases in the root deps.edn file. If I remember right, the dependencies specified in :deps will also affect aliases so that they are merged into them, but I'm not sure! At least this was the reasoning, to keep things separate.

Yeah, well, that’s what I expected. The thing is I can’t clearly see, apart from the profiles case, how this helps to isolate things. We tend to always have this :dev alias enabled by default and, at the same time, I can’t recall a single case when I needed it disabled. What’s to profiles, I don’t know (yet?) about any way to have such components under the :deps key with others and excluding these deps in non-`+default` aliases.

What I’m trying to achieve, btw, is some way to have two Polylith workspaces, one nested inside the other (the root one), working from the root one and being able to have a complete development classpath. The problem is, the inner workspace’s aliases cannot be enabled when doing so. (Or, at least, I didn’t find the way to achieve this over the years.) I guess, this is the same reason behind having all the components’ deps listed under their :deps keys. P.S. I’m okay for the poly tool to only work for each workspace separately.

I came up with the following workarounds for this: • move all deps of the inner workspace out of :dev alias under the :deps key — then I’m kinda limited with profiles use in that workspace, but otherwise, I think, I’m good to go with this approach; • add a :deps/prep-lib for the inner workspace that builds a JAR file and put this on the classpath instead of the plain source code — this kinda sucks, because it “librarifies” the whole thing and undermines the usual development workflow, so I opted out of this approach for now; • somehow use the :deps/root option — which I never tried before and thus don’t quite get how it works yet, and there is also an https://ask.clojure.org/index.php/12913/using-deps-root-with-local-root that prevents to do this with a :local/root dependency that the inner workspace is in this setup.

Would appreciate any thoughts on this problem. Maybe Mr. @seancorfield, if he pleases, can shed some Clojure CLI/Deps wisdom here? 🙂

Why do you have one workspace nested inside another? Stepping back, what problem are you trying to solve here? You are correct that tools.deps does not support transitive aliases (although you can programmatically build a basis with aliases so you can sort of workaround it if you want to write your own dependency management).

Okay, some background. I look for a workable way to split one giant monorepo into multiple workspaces, most of which are for end products (with deliverables), and the remaining code is allocated to the common platform components. Everything went just fine until one of the components became so big and complex (internally) that I felt an urge to polify it as well. Splitting it into smaller components is a no go, since it is a small framework that is supposed to be used as a whole. Moreover, to make things worse, it has its own top namespace and this is intentional, since I want to open-source it at some point. Having it a separate workspace helps to achieve this goal as well. (Anticipating a possible question about dependencies on other components, I found a working way to avoid problems with their versioning.) For my undertaking, a Polylith support for nested and/or interlinked workspaces (now dropped) was a solution. Now I’m trying to achieve this, at least partially, in a handicraft way with plain Deps and some scripting around it. So I would like to know more about the programmable approach you propose. Do you have any articles or repos that cover this topic in more detail?

No, this is definitely in the "out there" territory so you're on your own, I'm afraid. I'm looked at "transitive aliases" a couple of times but it isn't a priority so I haven't dug very deep. I don't know whether you can implement it entirely with the API presented by tools.deps or whether you'd need to fork the repo for this...

My only other comment here is that Polylith is intended for applications, not libraries, so turning your to-be-open-sourced "library" into a Polylith workspace is probably not the right way to go...

this is definitely in the “out there” territoryI meant, building a basis programmatically as you proposed above. Isn’t it something supported by tools.deps? Or did you mean something else, say a tools.build basis?

Polylith is intended for applicationsYeah, I know, it’s fancy, but I already managed to achieve this just fine, by exposing an API for the whole thing as a single base in the inner workspace and building it from the outer one with some shenanigans that copy-rename-carve parts of dependent components. Since both workspaces are parts of a single-versioned monorepo, this works, no problemo.

Still, I’m quite a newbie when it comes to hacking around existing Clojure tooling (Lein kept things a bit simpler easier than Deps/CLI/build, imo.). Wonder what you had in mind when wrote this: “although you can programmatically build a basis with aliases so you can sort of workaround it if you want to write your own dependency management”. @seancorfield Can you, please, give me one more hint on this direction?

https://clojure.github.io/tools.deps/#clojure.tools.deps/calc-basis -- you can merge-edns across many deps.edn files. Like I say, I don't know if you can implement transitive aliases directly based on that API. It's been a long time since I dug into this and I was originally approaching it from the internals of tools.deps since I wanted to see if there was some path to a supported way to do this, rather than just hack together support on top of it. Your situation is made more complicated by Polylith not actually using tools.deps for a lot of what it does with deps.edn files but it sounds like you've already diverged from what the poly tool provides -- I don't know how far down that path you've gone.

Aha! Started looking into the tools.deps sources and playing with it locally. Hopefully, it won’t get too long to hack it and use a patched version that replaces the basic one for this project. Kudos, Sean! You are one in a million!

Perhaps the last thought I would like to share and emphasize is that the problem, imo, is not in classifying or not classifying Polylith as an architecture. The problem is communication (in a sales sense) to potential users. That’s why, to solve it, I’d think in sales terms. > General guidance on the categories of components you should create and what responsibilities should these categories have. This is what I mean when I say a “logical layer” (of an architecture) or (its components’) “semantics” that things like P&A or CLEAN prescribe. Polylith in this regard is “semantic_less_”. Which is a great trait, since it means that it composable with other “architectures”, and thus is a selling point. > I don’t think we would even have considered adopting polylith without the tooling support. Probably, this is a unique selling point (USP) that we tend to overlook. “You may call Polylith an architecture or not, but the cool part is that it comes with a daily tasks automation tool that takes the responsibility of preserving invariants off your shoulders”. Without the poly tool we end up with a regular boring and “heavy” (to reason about, to test, to maintain the consistency of, etc.) clojure.deps monorepo. In some sense, it puts Polylith closer to things like Bazel and Buck than to all these fancy “architectures”. > How I sold polylith to my team and its surroundings was by focusing on how it enables a convenient monorepo experience… Same here as well, both for me and for some fellows of mine who also leverage Polylith for their systems. Given how hot this monorepo topic has been in the recent years, I’d rather highlight exactly this — “organizational principles and tooling for your Clojure/Python monorepo” — in the sales pitch. It’s always easier to sell what’s in high demand. P.S. I guess I was a traveling salesman in my past life. 😅

I agree with Mark: the structure/organization and the tooling are Polylith's actual selling points for real world users, not that the Polylith team thinks it's an "architecture". I think you'd get a lot less pushback and puzzled looks, if you dropped the "architecture" aspect of it all. You keep saying Polylith isn't a "tool" and you can use Polylith without the tooling support... but benefits like incremental testing and dependency analysis/visualization are big benefits and several people have said they'd never adopt it without the tooling. The main reason I was able to sell it at work was the tooling and, in particular, the incremental testing, and the ability to automatically build & deploy only the artifacts that are needed based on what's changed.

It feels like we would benefit from highlighting the poly tool more and pushing its benefits. One idea is to move more of the documentation to the poly tool, and to shorten the high-level documentation. I think that the existing high-level documentation is a little too academic in some cases and that a lot could be removed and rewritten.

Do you have that translation posted somewhere we could take a look? My French is a bit rusty these days.

Unfortunately not. Google translate couldn't translate the whole page, so I had to manually translate one post at a time.

I think the original is english, just remove the ?tl=fr at the end https://www.reddit.com/r/programming/comments/1ehk3d3/polylith_a_software_architecture_that_applies/

OMG. Reddit being Reddit.

Okay, thanks! Just followed a link!

I still think it's better if someone other than me can answer, e.g. @seancorfield!

Ah, yeah, very much Reddit just being Reddit. Ignore it. Definitely not worth trying to respond.

You are probably right.

Reddit is a very strange place... r/Clojure is better than most of the rest of it... r/Programming is a quagmire...

proggit is a pretty mediocre place

I don’t regret leaving reddit behind after the APIpocalypse

And now they’ve made themselves even less relevant by not showing up in search results (unless you still use google)

There's an interesting comment there, though not really poly-related:

Yeaaah… This one really pissed me off yesterday. I’ve seen a lot of Clojure codebases in the recent years, big and small. What I’ve noticed is that when a codebase grows and, especially, when there are multiple “growers” working on it in parallel, things ef up rather quickly, say in 2-3 years on average. And good luck to that poor fellow who’ll come on to maintain this junkyard eventually! (It’s usually me, lol.) This is exactly the Polylith structure + tooling that helps to maintain the “high-enough standards” that that guy/gal have mentioned. And, what’s even more important, do this *effort*lessly. At least, for the main part.

@mstoyukhin great point. I still hold my unpopular opinion that polylith is not an architecture per se but rather an enabler of architecture (a framework?) and its most important assets are its structure (conventions) + tooling just as you wrote. These make it easy to decouple module composition from deployment units and develop the entire ecosystem comfortably in an incremental fashion, reducing breakages. I honestly think focusing poly docs on these would help outsiders understand its benefits better.

Agree, @imre. I also sometimes think that the word “architecture” is too strong and vague when it comes to describing Polylith. People tend to think of architecture as of something macroscopic in the first place. “Architectural framework” is a reserved word, imo, for an even bigger thing, smth. like MODAF, so no luck here. I’d say that Polylith is rather an “architectural pattern”, since it is comprised of multiple well-known software design patterns and principles it follows. Finally, taking into account Polylith’s exclusiveness to Clojure (and, meh, Python) forming its sole development context, I would even consider it an “architectural style”. But none of these clarifications seem to help here as well. > I honestly think focusing poly docs on these would help outsiders understand its benefits better. Maybe focus on the outcome instead? And leave the architectural part unsaid? Like in “Polylith is a tool that enforces a set of well-known and time-tested architectural patterns and organizational principles to your Clojure/Python codebase.”

I’m away from keyboard, but will try to reply shortly. Polylith is not a tool. It can be used without tooling. It’s not just an organiser for other architectures. The naming and structure conventions are there to improve the UX when working with the building blocks, and to enable tooling support. It adds value, but could be removed. If we renove a couple of more things, then we would get closer to the Hexagonal architecture.

> I’d say that Polylith is rather an “architectural pattern”, since it is comprised of multiple well-known software design patterns and principles it follows. I would say this is true for most software architectures out there. Maybe software architectures like Clean architecture, Ports & Adapters, DCI, Onion architecture, Polylith, and others, should only been seen as software design, which solves different design problems, such as increased readability, changeability, and reuse. What's interesting to me is what problems they solve, and how well. Polylite comes from the functional world, which understands some basic principles to reduce complexity, such as separating functions and data, using immutable data, and trying to have as much side effect-free code as possible. While other architectures make a big deal out of organizing business logic in a certain way, this comes naturally from using components in Polylith. Without thinking about it, it is usually almost right from the start, and if not they are easy to split or merge, while you don't have to think about layering. Each component solves a small part of the problem, which can be used by other components to solve problems at a higher level. Interfaces and components are pretty much the only thing needed. So for me, we might as well turn the problem around and ask ourselves if we need the other architectures (yes, the question can be seen as provocative!). What does Ports & Adapters add that you don't get if you organize your subdomains into components? However, the list of things that P&A does not address is quite long compared to Polylith. That's why I wonder every time someone says that e.g. P&A is the solution, when it doesn't provide the same level of flexibility either in how the code should be run in production, the developer experience, or how well it addresses complexity. > I honestly think focusing poly docs on these would help outsiders understand its benefits better. I agree that we have not fully managed to convey what problems Polylith solves and what differentiates it from other architectures (or whatever we want to call them). It's probably more rewarding to compare it to P&A, The Clean Architecture, and the like, instead of comparing it to Microservices and Monoliths. The last two can be organized/designed in all three of these ways (P&A, CA, Polylith). We (the Polylith team) chose to call Polylith an architecture, because it largely solves the same problems as other software architectures, but I'm not the person to judge whether any of them deserve to be called architecture.

I think where people tend to criticize Polylith is that it isn't very proscriptive: you can still write your actual code any way you want, as long as you organize it into bases and components with interfaces. Polylith doesn't force you to do anything in terms of what I would call "architecture" so I really have a hard time with Polylith being described as an "architecture"...

P&A is a lot more proscriptive, as are most other things commonly called "architecture".

I don't agree that P&A is much more proscriptive, it's just organized in a different way and with different names. But yes, we are not explicit when it comes to the fact that business logic is usually best off living in its own components. We could provide more guidance on how to use Polylith. Many of the "architectures" out there, Polylith included, should maybe be called something else. We just looked around and saw that similar solutions were called architectures, so we went that path.

> you can still write your actual code any way you want, as long as you organize it into bases and components with interfaces That’s the thing I love the most about poly. Still, I miss this “logical level” from time to time, when it comes to grouping related things or just physically placing them close to one another. That’s why I will insist on a feature that allows to have your components in sub-dirs. For now, I just opt out to use fancy Unicode characters, e.g. ꞉ (he he, trust me, it looks like a colon*)*, in components’ names to achieve this.

> Many of the “architectures” out there, Polylith included, should maybe be called something else. An architectural pattern maybe? So not to argue with outsiders too much. Idk, whether it will help though.

> P&A is a lot more proscriptive, as are most other things commonly called "architecture". Yes, most other "architectures" are more proscriptive, like the https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html (P&A was maybe an exception). But are we sure that CA (as an example) is the best way of organizing code and to think about code? Maybe Polylith with a few more guidelines is what we need! 🙂

> deserve to be called architecture I don't think being called architecture or not defines the quality of any paradigm or methodology or whatever. None of my comments are out of disrespect or contempt. It's just an observation of what problems polylith solves for me and how. > where people tend to criticize Polylith is that it isn't very proscriptive My remarks weren't even criticism at all, in fact, just as Mark writes, I think it's a great asset of polylith that it doesn't force any particular architecture on the developer. > as long as you organize it into bases and components with interfaces > + > naming and structure conventions are there to improve the UX when working with the building blocks, and to enable tooling support. It adds value In my observation and for my goals the most important added value of polylith is precisely the (extendable) functionality provided by its tooling, as I wrote earlier. > but could be removed I don't think we would even have considered adopting polylith without the tooling support. > Maybe focus on the outcome instead? Precisely my point. One repository (atomic commits), as many deployables as you want, incremental testing, convenient component extraction and sharing. No restrictions on module responsibility or granularity. Can use any architecture you want, including no upfront architecture. In exchange the only slight inconvenience is that you need to adopt a few patterns as Sean mentioned, which might seem somewhat unorthodox. All in all, an easy sell to any development team that struggled with managing multiple deployables with varying levels of coupling and code sharing. You bend the shape (and not the responsibilities) of your code so it fits the conventions and you get the monorepo management benefits of the tooling. This is what makes me view it as a framework (it isn't a swear word!). Not in the RoR sense that the outcome is a web app, but a framework for managing your code->deployables journey conveniently. > this comes naturally from using components in Polylith Never underestimate the incompetence and disinterest of developers (as stated above). Writing usable code is hard, writing reusable code is even harder, no matter how easy it is to split code. Assigning responsibilities well is hard, no matter how easy it is to create new components. It's very easy to mess up even a simple standalone app where you don't even have inter-process dependencies. Done it many times. Future me will probably feel that I also did it with polylith-kaocha, despite it being developed in a poly workspace. > What does Ports & Adapters add that you don't get if you organize your subdomains into components? General guidance on the categories of components you should create and what responsibilities should these categories have. > every time someone says that e.g. P&A is the solution, when it doesn't provide the same level of flexibility either in how the code should be run in production, the developer experience, or how well it addresses complexity P&A doesn't prescribe at all how the code should be run in production or the developer experience. It addresses complexity by its guidance on assigning responsibilities. > P&A, The Clean Architecture, and the like By the way, given that in my observation the main difference is wording, detailedness and emphasis, I consider P&A, Onion, Hexagonal, and Clean primarily just different names for the same thing. > If we renove a couple of more things, then we would get closer to the Hexagonal architecture > + > functional world, which understands some basic principles to reduce complexity, such as separating functions and data, using immutable data, and trying to have as much side effect-free code as possible I read this that the Polylith authors recommend following good functional programming conventions and practices even on a component level, and recommend an overall architecture along the lines of Hexagonal. I don't think there's anything new in this and none of these are enforced by polylith's tooling, and again I consider this a good thing. Devs don't agree on how "functional" code needs to be and I'm sure the usefulness of things like P&A is also not universally accepted. But since poly doesn't enforce any of these, you're free to do as you please and still get the benefits I mentioned above. > we are not explicit when it comes to the fact that business logic is usually best off living in its own components. We could provide more guidance on how to use Polylith > + > Maybe Polylith with a few more guidelines is what we need? I'd focus instead on the flexibility. "we do recommend a b and c but you're free to design your system your way". > it largely solves the same problems as other software architectures > + > probably more rewarding to compare it to P&A, The Clean Architecture, and the like, instead of comparing it to Microservices and Monoliths I disagree here. By providing good monorepo tooling, polylith helps decouple module (component) composition/dependency management from deployable assembly while still allowing atomic changes across the entire ecosystem. To me this is completely orthogonal to what P&A&co address. It's somewhat closer to the Monolith/microservices problem in that these are deployment strategies and polylith allows you to assemble your deployables to your liking. You can deploy both from a poly workspace actually. You can have projects in a workspace that share almost everything and you can have projects in the same workspace that share absolutely nothing. How I sold polylith to my team and its surroundings was by focusing on how it enables a convenient monorepo experience through its tooling while requiring (enforcing) little change to migrated code and deployables, and leaving complete freedom in evolving the codebase later (as long as you uphold its few conventions). Naturally, you're free to call it an architecture and compare it with any pattern you think is fitting. I'm just sharing how we distilled its aspect that is important to us, and for which I've yet to find a better phrase than framework. Perhaps (also) presenting this aspect might help drive understanding in others who have dismissed it so far.

> I don't think we would even have considered adopting polylith without the tooling support. Same, to be honest. > How I sold polylith to my team and its surroundings was by focusing on how it enables a convenient monorepo experience... Yup, same here, really. The one big benefit for us of Polylith's "enforced structure" -- the flat components tree and the consistent <top-ns>.<component>.interface naming -- is that we are more careful about naming reusable code and we are more cognizant of "what one thing" should this reusable piece of code do that aligns with that name? Those are now discussions that we actually have, that we previously could avoid by having a fairly arbitrary organization of code into "trees" of namespaces.

We haven't changed our "architecture" at all by adopting Polylith, which is why I don't see it as "architecture" really. Our code is better organized now. We have incremental testing now (OMG, that is a huge practical benefit for our CI/CD system!).

I guess there's a difference between architecture as a description of how your solution is structured, and architecture as a description of how your code is organised, and we tend to conflate the two

This is a very good discussion. The most important thing for me is that we communicate which problems Polylith helps to solve. I agree with most of what you write. I think Polylith's decoupled building blocks are the basis for writing simpler systems. With that as a starting point, you can then add additional architectural features such as that services can be deployed independently (Microservices) or that parts of the code can only contain business logic (Ports & Adapters + Clean Architecture).