This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-03-25
Channels
- # announcements (2)
- # beginners (30)
- # biff (12)
- # calva (2)
- # cider (62)
- # clerk (18)
- # clj-commons (20)
- # clojure (9)
- # clojure-europe (6)
- # clojure-switzerland (1)
- # core-logic (6)
- # datomic (4)
- # events (3)
- # fulcro (1)
- # membrane (15)
- # off-topic (16)
- # portal (24)
- # practicalli (2)
- # releases (1)
- # rewrite-clj (45)
- # shadow-cljs (14)
- # tools-build (1)
- # xtdb (4)
Today I watched "Introduction to 'Atomic Architecture'" from here https://www.youtube.com/watch?v=5ygrCpC0axM . It randomly got suggested in my YT feed. I've been working on a very similar design / prototype for quite a while now. It has the same broad strokes, just maybe different emphasis here or there and different labels/names. I always referred to it as a Hub or Mothership (scifi reference) personally. It emerged from trying to attack problems with code maintenance of dozens of websites and applications. But I also want to improve the overall dev workflow, the robustness of data, the reuse etc. I started asking questions like "which parts are changing often", "who is using what", "how often are certain parts used and changed" etc. I flip flopped around certain ideas but it gravitated towards what is here called Atomic Architecture.The core is changing slowly. It has to be steady and stable, few people interact directly with it. The stuff that gets produced at the edges is much more fleeting, seen and touched by a lot more people. Our users, their users, crawlers, the whole world, so it needs to change and adapt more frequently. When I saw the talk I got excited and explained to my GF that the same design comes up in other places. She then said "Well it's more like a biological cell isn't it?" I love this metaphor. A cell also has a nucleus, which is where all the information is stored in DNA. And there is RNA, enzymes... a whole chain of metaphors for this kind of design, that fit like a glove. What else can be extracted and learned from the cell metaphor? It reminds me of how Alan Kay talked about Object Orientation, if I'm not mistaken he was trained in math and biology. But his fundamental idea was that it is objects all the way down. Hubris! Cells are highly complex, rich systems. If anything comes even close to what a cell is, it's a whole system architecture and not the little function calls we make and pieces of data we pass around. Those are just bits of information and processes. The whole thing however very much feels like a cell-like structure.
One day I want to dive into the field of complex adaptive systems (CAS) and apply that to code structure and organization. Main idea is there are emergent properties from simple rules and behaviors Good example may be Erlang/OTP?
I love the cell analogy. That's a really good insight and analogy that easily conveys the ideas , especially about it being a self-contained complex unit. The 'atomic' word is a nod to both atoms in chemistry (with electrons being the stateless apps) and the semantics of the clojure.core/atom or reagent/ratom. Atomic Architecture is the result of a number of conversations I've had with Clojure devs and architects over the past couple of years, and I'd love to discuss with anyone interested, just send me a DM, I'll also create a channel. It's partly a reaction to the dominance of microservice architectures which have become almost the default but not always the most appropriate architecture for a given situation. Btw. There's now a blog article on the matter which explains AA from the perspective of a set of principles. https://www.juxt.pro/blog/atomic-architecture/
I was just reading through the blog post and had a few comments. I really wish each section had a 1-2 sentence summary at the beginning. For example, it seems like the main idea of "Shared State" is the last sentence of the 4th paragraph. > But we propose a policy that all such services, within a given domain, source and store their state in a single centralised service. I'm not totally sure that's the main idea, but either way, it would be nice to start with the main idea and follow up with the explanation. I realize that the paragraphs flow from one to the other (which is great), but starting each section with a summary makes it easier to skim and use the article as a reference (in addition to telling a larger story). Under "4. Data Consistency", it uses the definition "In short, consistent systems are systems that users can trust. They handle data with care, accuracy and precision." I actually really like the breakdown that is commonly referred to as https://en.wikipedia.org/wiki/ACID. It seems like the post uses consistency to refer to both consistency and isolation, but doesn't mention the atomic or durability parts of ACID. It doesn't seem like failures and errors are given much discussion. Usually, that's the hard part of building systems that is hard to retrofit.
I'm pretty skeptical of the cells analogy. Not because it can't provide a good analogy for a system architecture, but because it's really easy for multiple people to agree on the cell architecture, think they all agree, but have wildly different mental models. I put it in the same category as FRP and MVC architectures. I know precise definitions for these architectures exist, but I'm not sure I've ever met two people that agree on what they mean by MVC or FRP if you try to break them down into actionable design decisions. That's not to say all design explanations are bad. There are many of examples of design languages that do convey meaningful ideas with precision: • The epochal time model • CSP • ACID • concurrency vs parallelism
Thanks @U7RJTCH6J - I agree - it would be better to summarize what each principle is first, before the discussion. The article isn't finished yet, so I can do another revision. (I published it by accident and then rushed to flesh out each point).
I like your point about failures and errors. While not mentioned in my article (yet) one of the drivers for this architecture is the inconsistency that results when you have failures/errors. Imagine you sell a product on your website. You send a message to your accounting ledger microservice to debit the customer's account. Then you send a message to the warehouse microservice to ship the product to the customer. Then you receive a failure from the accounting ledger microservice. What do you do now? Do you send another message to the warehouse to try to cancel the shipment? This is one of many examples where complexity arises from the lack of atomicity in the system. When your have an atomic system (the A in ACID), then either the accounting ledger and product shipment succeed together, or they both fail.
This concept of atomic architecture says you can and should co-locate the state of the warehouse with the state of the accounting ledger. So a business operation called, say, 'Buy Product' would simultaneously check the account exists, there is sufficient credit, the product exists, the warehouse has sufficient stock to fulfill the order, all in a single transaction. If all these checks are OK, then the account is debited, and the stock levels are adjusted, in a single state change. This is possible because all the state is centralized in such a way that it can be subjected to an atomic change. I'm not saying this necessarily implies a single RBDMS database (although that might be sufficient in many cases), but there is a central store of data (even implemented as a cluster of data sources) that semantically behave as one, at least to external clients/applications.
I still like the cell analogy, although @U01EFUL1A8M’s mention of a 'hub' is a good choice too. As long as people understand the point that the cell has a nucleus containing the information for the cell. I'd like to hear some of his other ideas around this analogy.
What I understand here is that the nucleus is not just an information storage and validator, but an actual coordination point. Just like the atom
in Clojure.
So the analogy makes a lot of sense when you think of the nucleus als holding things together. The electrons are drawn to the center, AKA held together and coordinated by it.
This is essentially the same pattern we apply to say UI programming (re-frame, fulcro etc.) where you have one graph storage at the center that the rest subscribes to. This global view of the world lets us coordinate all the state in a consistent fashion instead of having it spread out on local objects.
And by the way whenever I don't do this consistently (in UIs), I end up moving state into a single place at some later point, because there's often some cross concerning coordination requirement that I haven't thought about earlier.
The comparison to re-frame and fulcro is useful. It's exactly that idea, that state is better being coordinated in a single place than spread across multiple objects, especially if you can coordinate change and embrace ACID properties.
I've shared this topic to #C0904S2QJ
late to the party. Another thing to learn from biological cells is that they reduce entropy within itself by increasing entropy of the outside system. This is how interesting structure form within the cell. I use this metaphor to justify keeping everything in a single namespace (low entropy) to begin with. As the namespace gets too complicated (increase in entropy), I reduce the entropy within the namespace by extracting symbols into a new namespace increasing entropy of the system by reducing entropy within the given namespace. The end result of doing this is there's high dependencies between the symbols within a namespace while adding small cost by introducing external dependencies (high cohesion within the namespace and low coupling between namespaces). Changes within the namespace (cell) have lower chance of creating a domino affect to the rest of the system https://www.quantamagazine.org/a-new-thermodynamics-theory-of-the-origin-of-life-20140122/
@U07SXG7DY I agree, but the atoms have already a structure, way of operating and it's consistent. This means that this practice can be dangerous in software development on a larger scale. If you take x people to start working like that, each can reduce entropy in a different way and expect a different structure, which will make the code hard to comprehend and reason about. (Imagine one person trying to implement clean architecture by reducing entropy and the other goes with regular 3 layers). So you need to define how you are going to reduce entropy with time.
yes different people will have different ways of reducing entropy which is fine. ultimately the things inside the cell has high cohesion (meaning high dependencies with each other) and the dependencies between the cell and its environment is minimized through a small set of interfaces. In biological cell, this small set of interface point is the geometry of the cell wall
> how you are going to reduce entropy with time Keep dependencies together. Extract dependencies into a new namespace only when that dependency is needed by other namespaces. Introducing a new namespace increases entropy of the system but reduces entropy within the namespace. Minimize the dependencies between namespaces. This fits with the mantra "high cohesion, loose coupling"
I love the cell analogy. That's a really good insight and analogy that easily conveys the ideas , especially about it being a self-contained complex unit. The 'atomic' word is a nod to both atoms in chemistry (with electrons being the stateless apps) and the semantics of the clojure.core/atom or reagent/ratom. Atomic Architecture is the result of a number of conversations I've had with Clojure devs and architects over the past couple of years, and I'd love to discuss with anyone interested, just send me a DM, I'll also create a channel. It's partly a reaction to the dominance of microservice architectures which have become almost the default but not always the most appropriate architecture for a given situation. Btw. There's now a blog article on the matter which explains AA from the perspective of a set of principles. https://www.juxt.pro/blog/atomic-architecture/