This is easily my favorite of the series. Making sleep be just yet another multimethod is a good insight! This episode addresses one of my blindspots, the intermediate architecture.

It's kind of funny how the final evolution of every complex system seems to be "state machine", and yet it's probably the last thing I ever think about despite covering the topic exhaustively in university

Great episode guys 👍:skin-tone-2:

Awesome! Glad you enjoyed it.

On my point about state machines, I think the reason is probably related to what you guys said this and last episode about stopping: for so many problems we tackle in software, this kind of thing is likely over engineering. That tends to have the effect to think of something like state machines learned in school as more of an academic exercise than a power tool. Or maybe it's just me

Totally agree. Knowing when to stop is key to providing the right value for the effort.

@U02PB3ZMAHH Thanks! I like how you put it: "intermediate architecture". Sometimes a tiny bit more generality does give you an outsized win.

@U04RG9F8UJZ Yes, it is a state machine in a literal sense, but a very specific kind. Some key things, for me: large-grained steps, clear distinction between "operations" vs "integrating results", and the state is 100% represented in the "context". A classic state machine can represent some of the state implicitly based on your "node" in the state graph. (Eg. A deterministic finite automaton.) I don't find those sort of state machines very practical for general process management. They can be great if you're implementing pattern matching, however.

@U5FV4MJHG @U0510902N Thanks so much for this last episode. I've been so excited about this topic and have been eagerly expecting this episode after that gruelling and long-lasting cliffhanger in the previous week and also after listening recently to episode https://clojuredesign.club/episode/024-you-are-here-but-why/. At the same time, the think-do-assimilate is one of the most challenging mind-shifts for me, so I though to try it out first with an artificial problem. I thought to try it out on some simple process, with mocked dummy IOs: get an information from two sources, compare it, and based on the result, send an email or send a Kafka message. To get something off the ground, I put together an initial version, see https://gist.github.com/mkrcah/8fac30c365069b6d1ab9466c04b9f470#file-tracer-bullet-clj, without the think-do-assimilate loop first. (Sorry if I missed some obvious stuff that you have discussed on the podcast already, I have yet to listen to all the episodes.) What I listened to so far and tried to incorporate was: • aggressively minimal single-entry IO, • pure extractors, transformers and predicates, • growing bag of data. • flat maps with rich keys Having the v1, I tried to refactor it to the think-do-assimilate, see https://gist.github.com/mkrcah/8fac30c365069b6d1ab9466c04b9f470#file-think-do-assimilate-v2-clj for my first attempt of ever doing this. Some first impressions so far: • Relistening to the episode, I think I got the fundamental principle right, yet I've been having some struggles (see below) • Yet, I love love love the purity. It feels as if the many pragmatic concepts that you talked about in the podcast all culminated together. I can feel that testability, reliability that you talked about. • Also, compared to v1, it feels there's very clear structure how to do things that I don't have to think about. • Also, having the full context at all times feels empowering and feels like it provides much more control and visibility, such as that straightforward ability to start in the middle. However, I have also experienced some struggles: • The overall logic felt harder to follow: ◦ In determine-operation, I initially made an incorrect condition and ended up with an infinite loop. I think I need to write much more of these to get a hang out of it. ◦ Compared to v1, the logic for one operation is now split in two places: determine-operation and update-context. This is something that requires some getting used to and is perhaps my biggest struggle. ◦ Thinking on this more, perhaps the struggle stems from the unfamiliarity of this mental model of think+assimilate. I sure want to get more reps in to see how it feels. • In update-context, I wanted to also capture the raw request, which was part of the operation. However, I started to feel a bit of a design tension when moving the data from the operation to the context map, such as (assoc bag :google/request (:request operation) . It feels like repeating myself and I'm wondering if there's perhaps a cleaner way to achieve the same. In summary, though, I'm in awe when thinking about this approach. Thank you both very very much for this fantastic episode 🔥

@U0609QE83SS Thank you so much for the DM and the delightful video you sent. Would you mind posting that here? I suspect many more people would get value out of it. I love the experimentation going on, and exploring different points in the design space. What caught my attention: you noted that in my code, I combined the “think and do” operations in that big do-it function. I did it that way for precisely the reason you mentioned: I wanted to keep the code together so it was easier for me to understand — interestingly, I got one state transition wrong. My analysis: splitting out the “do” part would have decoupled it, but also it would reduce coherence (i.e., the degree to which the components work together as a logical and cohesive whole). I chose to keep it together for now. And hilariously, like you, I also got an infinite loop on my first couple of attempts, because of getting the final state wrong. 😂 Nice work!

Something that crossed my mind: I occasionally wish there was a way to keep the pieces a little more more tightly coupled — so that misspelled keywords wouldn’t result in entire thing falling apart.

Hi @U6VPZS1EK Yes, absolutely, here's the short 2min Loom video https://www.loom.com/share/0ead62a67a1d4846acd0bce5dc085d78 I love that experimentation and sharing as well. Thanks for expressing that. For coherence - yes, I like how you describe it. I think I need some hammock time now to let these approaches sink it, it's been quite a shift. Also want to listen to episode https://clojuredesign.club/episode/024-you-are-here-but-why/ again.

I kept thinking about that coherence struggle, as @U6VPZS1EK put it, where the assimilate + think function is split into two places. I listened again to 024, and in that episode, they merge assimilate + think fns into one function, called decider. The IO is still kept minimal on the edges with the other function. I tried out this approach on the synthetic example and my first impressions are really good. The coherence is there + IO is fully on the edges. See the example code https://gist.github.com/mkrcah/8fac30c365069b6d1ab9466c04b9f470#file-think-do-assimilate-v3-clj-L70-L127. I tried out to make a generic orchestrating function as well so that I can it various workers and deciders. Initial, maybe a bit clunky, version turned out like this:

(defn execute [init-bag worker decider]
  (loop [bag init-bag
         operation {:kind :start}]
    (let [updated-bag (decider bag operation)
          next-operation (-> updated-bag :operation)]
      (if (= :done (-> next-operation :kind))
        updated-bag
        (recur updated-bag (worker next-operation))))))

Awesome, @U0609QE83SS — you might be interested in the epic awesome rewrite @U04V5VAUN did on my code in this thread: https://clojurians.slack.com/archives/CKKPVDX53/p1709561163370489?thread_ts=1708991492.816989&cid=CKKPVDX53 I think it’s 10x better than my ending point (which was 10x better than my starting point!) He even posted a 5.5m video of his refactoring process, to address my comment of, “didn’t that take a lot of typing and repetition?” (Answer: no! 2.5min, and when I watched it, my response was, “I need to do that!“)

@U6VPZS1EK, thanks for the link, I'll check that one out 🙏

I've been thinking about coherence a lot as well. Merging the assimilate and think into one function is a great idea. The other is to make the assimilate very straightforward, i.e. just assoc into a :result key what the "do" function returns.

@U05254DQM I'm happy to hear you're enjoying it! I love the pivot idea! I'll have to keep cat videos in mind for the future.