Fork me on GitHub
#architecture
<
2023-10-07
>
tengstrand11:10:29

Hi everyone! I've been working on a blog post for a week or so, that describes the Polylith architecture, by comparing it with the https://en.wikipedia.org/wiki/Hexagonal_architecture_(software) architecture (also known as Ports & Adapters). Before I publish it, I would like to get some feedback, to see if people understands it and to fix English mistakes (that I tend to do because I'm not a native English speaker). Please register your interest in a thread here, or message me on slack, and I will send you a link. You will need a https://medium.com/ account to add comments.

Rupert (All Street)13:10:51

I’d be interested to review the draft.

tengstrand15:10:28

That sounds great! I've got feedback from elsewhere, but when I'm finished with that, I will send you a link to the blogpost.

seancorfield15:10:56

I could probably read over it today or tomorrow and provide feedback.

tengstrand16:10:02

Thanks Sean. Will send you a link as well.

Thomas Moerman07:10:11

Also interested. My current architecture uses Polylith, and tries to approximate a hexagon-architecture, so I'm very curious about your perspective.

Thomas Moerman07:10:02

Don't have a paid medium account though.

tengstrand07:10:31

Sounds good @U052A8RUT. I'm still working on fixing the feedback I've already got, so will come back to you when that is fixed. I don't pay for Medium either, but I have an account.

👍 1
fuad11:10:23

I'd be happy to provide some feedback

ozzymcduff19:10:23

Sounds interesting. How does it go?

tengstrand20:10:51

It's going pretty well.

tengstrand08:10:31

Hi everyone! Now I have updated the blog post. Please have another look, and come with feedback!

tengstrand08:10:20

The English hasn't been checked yet, but this will be done by an English teacher, so you can skip commenting on minor grammar and spelling mistakes! 🙂

tengstrand03:10:30

The "Code sharing" and "Code structure" will probably be rewritten or replaced, so you don't have to give feedback on these right now. Or just wait with feedback till I have updated the whole thing!

tengstrand06:10:39

I have made a lot of updates to the not yet published blog post. Please have another look, and add comments directly in the document! Thanks. polylith

tengstrand21:10:28

Has anyone here used the Hexagon architecture (Ports & Adapters)? If so, how many applications (services/tools) did you have? The Hexagonal architecture says nothing about how to structure the code. Where did you put the code for each application? All applications in a single src directory, or one src per application? How did you structure each application? One source file per concept, or something else? Did you use a monorepo? Are there any advice on how to structure the Hexagonal architecture out there?

tengstrand21:10:47

For example, let's say you store your whole domain in one place. If we have ten different services, do you copy the whole domain into each service, together with a suitable incoming adapter, when you build and deploy each service? Or do you see the codebase as ten different domains that live in ten different src directories (ready to be built)? If so, I would be very surprised if we didn't end up with some degree of code duplication when dividing a single codebase into ten.

fuad11:10:57

I've worked with hexagonal/ports-and-adapters quite a bit in the context of (several hundred) microservices where each microservice is an "hexagon". Here's an anecdotal description of what it looked like: In that scenario it was relatively easy to do so because there was very little sharing between services when it came to domain logic: each service encoded a bit of domain logic and delegated the rest to other services. There was a lot of sharing when it came to infrastructure (networking, authentication, authorization, schema contract verification etc). This was done via libraries in a private maven repo. When it came to the shape of the hexagon: • secondary/driven ports usually represented infrastructure concerns such as networking and interservice communication (http-client, kafka producer, databases, observability and dependency on external third parties. • secondary/driven ports were encoded as clojure protocols (e.g. http-client port, third-party-service-client port) • adapters for secondary/driven ports were encoded as implementations of the above protocols (e.g. clj-http-client, http-kit-client, stub-http-client, stub-third-party-service-client, etc) • it was quite common for driven ports to have at least one stub adapter so that no external infrastructure was required during development/tests and so that tests could verify calls to the ports • primary/driver ports were usually tied to the communication/networking mechanism (e.g. http server, kafka message handlers) • primary ports were written as boundary functions (pedestal handler functions, kafka consumer handler functions) • primary adapters were implemented as stateful components binding the external world to to the primary ports (e.g. jetty binding to a pedestal router, the kafka consumer binding to the message handlers, etc). • the whole thing relied heavily on stuartsierra's components: ◦ primary adapters were components that would bind to networking ports on start. They would would take the ports (e.g. the pedestal router) as an initialization parameter ◦ secondary adapters were components bound to certain keys in the system map associated with the ports they represent. For example, the system map could have a key :email-sender that would be assumed to point to a component implementing the EmailSender protocol (email sender port). Now, this whole thing is missing the actual domain logic and orchestration: this was done by simple clojure functions, either pure (logic functions) or impure (controller functions). Controller functions are called by primary ports to fulfill the use cases and they take the ports they would depend on as a param (e.g. a controller might need to send an email so it takes a map of dependencies with the :email-sender key). The controller coordinates interaction across multiple ports and side-effects and call logic functions as needed. Controllers and logic functions are each organized in separate namespaces separated by "subdomain", although most services only a handful of such namespaces. This worked really well in that context where each service did very little, so the only structure that was needed was separating the boundaries from each other and from the domain logic of that service. --- More recently I'm working in a different context where I have a single, monolythical codebase and I'm applying the same principles as above and they have been serving me well. However, I did notice that, with a large monolyth, there's a greater need to clearly separate the multiple domains inside of the application. In this scenario, the dependency between domains is encoded as dependencies between functions in different namespaces instead of between services with network connections in between them, so a lot of the work that was previously done in the boundaries now needs to go "inside of the hexagon" (there's an inflation of controllers and the dependencies between them) I'm still trying to find the right way to model some of these things, but up to this point what I have adopted is something similar to what polylith promotes: each subdomain contains an interface namespace (in my case I even split the interface in two, one ns for queries and one for commands) and a lot of smaller namespaces with private implementation details. I still use clojure protocols and component implementations as described above in order to easily swap implementations (either in production or in tests). But this remains almost exclusively restricted to infrastructure-related interfaces and third-party dependencies. It's interesting to note that these adapter components are usually the things I make assertions against when testing side-effects. I've been considering the idea of using clojure protocols to encode the interface of domain logic components as well (in addition to the interface of infrastructure related things such as networking and dbs as described above). This would allow me to test a single "subdomain" by injecting stub implementations of other subdomains and making assertions against these stubs, but I'm not sure I like this idea. --- In summary: I don't think the hexagon gives you strict directions in how to structure your code other than encouraging the separation between the boundary code and the internal logic (so that they both can evolve separately from each other). In a small codebase (e.g. a microservice) this might be enough structure. In a larger codebase, where the core of the hexagon is much larger, some structure is needed to properly organize business logic and domains/subdomains. The https://alistair.cockburn.us/hexagonal-architecture/ makes very little (perhaps zero) statements about that.

1
👍 2
fuad11:10:11

I'm happy to clarify any points or provide some more concrete examples if you're interested. I've been meaning to do a writeup on this subject for a while and this was a good rehearsal 😅

sonnyto16:10:58

your questions are unanswerable. My heuristic is to keep things together in one module until it needs to be split up. Start small, start simple. Don't think how many ports or adapters or layers you need or where to put them before you even write a single line of code. These are emergent properties that cannot be predicted before you start. The justification for this is keep entropy low. only increase entropy of the system by moving code to new modules if it decreases entropy within a module. Over time the entropy of your systems will naturally increase (more modules). This is fine as long as the entropy of within the modules are low. The dependencies between modules must be minimized. This is what biological systems do. Life exists because it minimize entropy with in it by increasing the entropy of the surrounding.

sonnyto16:10:54

my definition of a module is loose. It can be a namespace, it can be a closure, it can be a data structure

💯 1
tengstrand02:10:23

Hi @U3RBA0P4L! What a great answer and summary of your experience with the Hexagonal architecture! It was exactly what I was looking for. I have a question. With those two different architectures you worked with, would it be possible to implement incremental testing per repository, based on changes in the version control system (e.g. Git)? Polylith operates at the source code level, and compose the artifacts at build time, and since the directory structure reflects the architecture and its building blocks, it is enough to check in git to see which parts are affected by the change and which tests we need to execute. So incremental testing is something you almost get for free in Polylith. I'm just curious to hear what you think, because this can be a hard problem to solve!

fuad13:10:44

I don't have much experience with incremental testing. In the scenario of the microservices+libraries, each "artifact" (a service, a library) was tested in isolation. Service dependency on library is specified with specific versions, so changes in the library aren't immediately propagated and tested in the service. The mechanism for avoiding outdated dependencies was to basically automate bumps at a certain time interval and some CI checks to warn about new versions being available. Service dependency on other services was tested at the contract level: each service publishes it's contract via CI so other services would pick up. The whole service-to-service dependency tracking used an in-house dependency graph assembled from the results of code analysis. Achieving this required a huge effort from horizontal teams at the company and are simply not feasible for smaller teams unless they leverage some open standards like OpenAPI perhaps. Even then I'm not sure how much can be achieved out of the box with those tools. It's important to note that in this case we did not use a monorepo, but every service/library lived in a separate repo. --- In the scenario of the smaller app I work on right now I haven't felt the need for incremental tests due to the size of the codebase: I can usually run all tests in under 5 seconds. I can see how that would become more of a pain in a considerably larger codebase. I do remember facing a similar issue (although not in the context of testing) in a different project I worked on recently: we had a monorepo with a few apps and a few libraries; the apps used the libraries by adding their paths to its :source-paths in project.clj. The trick in that case was: when do we trigger the deployment of a certain app? Basically we had some a script that would detect changes in libraries (the list of libraries specified in the script itself) and trigger a deployment to all apps, regardless of wether they used them or not. Besides being a bit wasteful (sometimes some apps didn't need to be deployed), it was error prone, since adding a new library to the monorepo and to the :source-paths wasn't enough: the script needed to be updated as well. In that case one solution we thought was to migrate to deps.edn so that the dependencies would be defined declaratively and in a way that would be easy to query mechanically so that the script would pick them up automatically. As I understand this is exactly what polylith gives you out of the box.

tengstrand14:10:41

Interesting. Yes, the value of incremental testing increases by the size of you system. It's not an easy thing to get right, but it's pretty trivial, or at least not rocket science, to implement as a Polylith tool, because of the standardised directory structure and the way of putting together "blocks of source code" into artefacts. So you basically only have to ask git and see which bricks and projects that are affected, and then test these together with indirectly changed ones. In Polylith, this is done by checking the "imports" (require statements), to get a dependency tree. Not trivial, but a lot easier than other ways of doing it.

tengstrand18:11:05

I just https://clojurians.slack.com/archives/C8NUSGWG6/p1698864153152019 a new blog post in the news-and-articles channel.