Clojurians Log v2

Somehow Nate and I seem to arrive at cooking metaphors. Perhaps we need to stop recording before dinner?

Right. It's interesting because on one hand it's exciting to work on green field since it avoids the pain of all kinds of weird constraints and rough patches, but on the other hand it's somehow easier to just stomp all over the green field and then fix it up than keep that manicured lawn throughout

Fun episode as usual with a pun-ishing ending. I enjoyed the metaphors and I really liked calling attention to "right sizing" your ingredients. Complecting things (cake mix) has a cost, but so does decomposition to the point of excessive granularity (wheat and cows). I've encountered similar when applying DI in OOP systems. At a certain point, you decouple the pieces to the point of losing cohesion and you have no real sense of what it's supposed to do or how the parts relate: everything is just wired by magic to everything else. While it's true that the wiring isn't "business code" and kind of a waste of time to write, it's also a kind of map or orientation that does provide value for navigation. What I'm wondering is: do you guys have any useful heuristics or concrete examples of how to "right size" ingredients? Of course it's always going to depend on context, but maybe there's some lessons you can distill from typical situations you've encountered?

Thanks for the feedback. Glad the episode and metaphor made sense.

I agree that it's hard to right-size things. It definitely is something that both gets easier and harder with experience. Easier in that there are some ingredients you're familiar with and use them in future projects, harder when there are new things that might look like other ingredients but should not be handled the same way.

@nardell, thanks for the link to the paper. It looks like a great read! Great functional composition is definitely about little units of clearly defined behavior being brought together into larger behavior. Side effects create obstacles to that. @john.t.richardson.dev, I know what you mean about what's old is new again. I remember the same language around "procedures" and "abstract data types". I think the more recent innovation of persistent data structures has allowed functional programming to be performant and scale up composition up to very large units of functionality.

Yup, with immutability, the typed walled garden that OO gives you isn't as necessary. I was just listening to Episode 42, which also blends into this, and is related to the "glue" idea in the paper. It seems to me that functional programming in Clojure has a virtuous relationship because • it has data-oriented entities - with no behavior attached (your example of a CSV file was instructive) • it has data-oriented functions - that allow any shape that conforms to the (implicit) interfaces (ie if the data has the fields I need, I'll do my job) • immutability ensures that the entities can't become corrupted • all of the above can be contained in modules, providing data hiding via scoping And also • spec gives you some way to verify that the runtime inputs use that implicit interface • spec also lets you test that your function behaves appropriately - rather than typing (bonus: it checks the function's behavior via generators) All this creates an environment safer than C and more flexible than Java, where you can glue code together.

very nice analogy with the cake mix (framework) vs ingredients (core fns / lib fns). It's a shame that Cake mix is so popular 😅

@john.t.richardson.dev Nice summary!

@nardell Good read about Functional Programming. I think that's how we do it with Typescript/Javascript nowadays - functions as glue code so you can compose things together. It seems like what was old is new again. I remember reading about this sort of thing back in the 90s, except using function pointers in C. I feel like Clojure makes this "gluing" easier with multimethods, as well as by being able to pass plain-old-functions around.

In this episode, when I heard complected, I thought "Java EE", or "Spring". They are so wonderful while the work fits in the ecosystem. Java EE containers would provide "batteries included" authentication and authorization, session management, templating engines, and so on that work together seamlessly. But they suffer from the last 10% trap. (https://deviq.com/antipatterns/last-10percent-trap). We would often skip that last 10% due to this trap. (While I haven't used them yet, my understanding is that the Clojure way is to provide libraries that you can mix and match.) Anyway, you guys were probably talking about a lower level (or maybe were being intentionally vague?). Maybe when you're building a reusable component? It's always harder to break things down into composable pieces. It takes several iterations for me, and a decent amount of thought. Great episode!

@jason.bullers I like your point about decoupling to the point of loosing cohesion. Larger pieces help you have a sense of what to build with it. Why make shelving system out of Legos when I can buy parts of a shelving system from Ikea? Each part wants to be part of a shelving system and points me in that direction. Legos could be anything. As to a heuristic, I work hard to keep I/O separated from pure. I also keep things in the same information domain together in the same file. Also, there are the natural kinds of splits: predicates, reducers, mappers, etc. Aside from that, I err on the side of making larger pieces until I notice a need to split them.

@john.t.richardson.dev We we trying to stick at a high level, so I guess that's going to cause a certain amount of vagueness! 🙃 I like your point about J2EE and Spring and the last 10%. I still have visceral reactions when I think of them! The complexity in those systems is awful! I like your point about finding your way to the right level of composition. @nate was touching on that too. It's hard to know in advance what the "right level" will be. I have come to believe that the "right level" moves as your software evolves. I worry lest about "big" pieces than I used to, since I can split those, and I worry much more about keeping I/O and side effects on the edges. I think the "cake mix" occurs when too much of dissimilar ingredients (like I/O and transforms) get all mixed together. The "cake mix" problem also makes me think of your earlier comment about building on top of frameworks.

@neumann I like the Ikea shelving analogy. One that I've used before also involves Lego: it would defeat the purpose to buy a preassembled and super glued kit, but it's also insane to build a huge thing with a giant bucket of flat 2 by 1s

It's hard to know what the right level of composition is from the beginning. Usually, you are exploring the problem and trying to figure out data shapes and features at the same time. It reminds me of the https://blog.codinghorror.com/rule-of-three/. Applied here, it means you don't know if something is the right level of composability until you've used it three times. Like @neumann said, something that's a box mix is ok at first, because it's easier to write, and after you need to use it for something else, you'll figure out how to break it apart to the right level. I often fall into the trap of trying to make things perfectly composable from the beginning. After waffling around for a while, I usually end up writing something that's more messy and box-mixy so I can understand the problem and then break it apart. Then I have understanding and composition and really start to pick up steam. Now, I can make some things more composable at the beginning because I've done them before, but there are always parts of new applications that I can't right-size at first. So I keep I/O separate from pure data transforms, try to keep functions doing only one thing, make functions in the "Clojure categories" (predicate, reducer, etc.), and generally give myself some slack for it being my first time through.

@nate "waffling around", eh? It all makes sense now...

The puns will always come around. 😉

I know what you mean though: I was just talking to some coworkers about something similar yesterday, when I was showing how I'd refactor a small GUI app to be testable. Someone asked how I generally work now (in Java) and I realized that for a long time, I was so focused on the "right" and testable design that it always wowed me to see what people put together at hackathons. I've sort of hit a middle ground now where I'll just hack something partially functional together, make seams to get it under test, then iterate the design and remaining features test-first

I'm sure big chunks of that are transferable to Clojure, just trying to build the intuition. That's why I listen 🙂

@nate The puns will definitely come around!

@jason.bullers I tend to get caught up in trying to figure out the "right" design too. I can spend a ton of time trying to think of all sorts situations. A habit I'm trying to develop is to make sure I have the major separations in place and just run with my first idea for a while. @nate has consistently encouraged me to get just something working end to end. I've realized that I actually don't understand the problem until I do, so I'm actually solving the wrong problem until I have that "tracer bullet" solution. One thing I love about Clojure is that I can usually go back and adjust based on what I learn without having to do big refactors.

Listening to this episode while making an espresso. Now I wish i could spend the day making biscotti from ingredients - a very composable cookie. In all seriousness, the episode would be a great intro to discussing the paper “Why Functional Programming Matters “ https://www.cs.kent.ac.uk/people/staff/dat/miranda/whyfp90.pdf.

The author describes functional programming as a new kind of glue, and the goal of the functional programmer is to write smaller, more composable modules that can be glued together in new kinds of ways.

I have had the experience a few times in Clojure where I added a new feature, with elegant use of function composition. A very frictionless experience.(NB. I am not a professional Clojure programmer)