Fork me on GitHub
#data-oriented-programming
<
2021-03-11
>
Yehonathan Sharvit16:03:01

I’d like to come back to the topic of loose coupling as promoted by Rich Hickey. I think I finally grasped what Rich meant by loose coupling. Let’s take the example of a 3rd party library that gives access to Google calendar API. Let’s assume this lib provides a function createEvent to create an event. Now let’s compare how we pass the information about the event we’d like to create: 1. In a statically typed approach (FP or OOP), we need to pass an Event record with the information about the record 2. In a data-oriented approach, we pass a map with the appropriate field name There is a huge difference between the two approaches: 1. In the statically typed approach, the information is passed in a way that is dependent of the implementation of createEvent (Rich call it a data concretion). The only way for my code to generate the information that createEvent expects is to use the Event record. There is a tight coupling between my code and the library code. 2. In the data-oriented approach, I am free to generate the information as I want. The only constraints that I need to respect are the field names in the map. There is a loose coupling between my code and the library code. @me1740, @cgrand what do you think?

cgrand17:03:07

bad example as it ends as a network call

cgrand17:03:53

it has been a long time since I listened to RH talks. that’s dependency coupling: two modules that share data shouldn’t have to share code, only agree on a spec (general sense).

Yehonathan Sharvit17:03:41

I don’t get why you say it’s a bad example

cgrand17:03:32

it’s a JSON msg in the end

benoit17:03:41

I’m not seeing the difference between the two to be honest (coupling-wise). In the Clojure case, your code still needs to pass a map with the right keywords. The main advantage of using maps over classes like Event is that you can reuse all the functions to work on maps to manipulate your event data record. Instead of creating a new abstraction for each type of record you manipulate, you reuse the same abstraction (Map). This is related to Perlis’ quote: “It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures.” So I agree with you that to represent static information, reusing the same abstraction integrated in your language is much better than creating a new class every time like you do in Java. But I wouldn’t “sell this” as a reduction of coupling.

Yehonathan Sharvit18:03:52

@me1740 Don’t you see a difference in terms of coupling between being forced to pass an instance of class vs. passing a map?

Yehonathan Sharvit18:03:03

How would you define loose coupling then?

andy.fingerhut18:03:54

There are older systems like CORBA and COM that I have not used, but my understanding is they much more tightly coupled the language used in the hosts with the messages being passed over the network. JSON / XML / EDN / etc. are already a way to help avoid that coupling -- you are passing values between hosts over the network. Any semantics of them being RPC calls or not is up to the application to define (or not).

andy.fingerhut18:03:04

JSON / XML / EDN don't by themselves impose any restrictions on what map keys are required, or optional, and what kinds of values they are expected to have. CORBA/COM/etc. style things do.

andy.fingerhut18:03:33

(is my very basic understanding of things like CORBA/COM/etc. from 10,000 foot view and never having used them myself -- corrections welcome)

andy.fingerhut18:03:50

Rich Hickey mentions and at least briefly discusses CORBA / COM in his talks "Value of Values" and "Language of the System", found quickly by grep'ing through his talk transcripts here: https://github.com/matthiasn/talk-transcripts

andy.fingerhut18:03:27

I don't have an example handy off the top of my head, but I'm nearly certain there are straightforward examples of using JSON for over-the-network communication between hosts, and still having other forms of tight coupling, different than the ones CORBA/COM get you into.

benoit18:03:31

For me "coupling" means the tendency for 2 pieces of code to change together. When a function calls another function, there is some coupling because if you change the spec of the function you might have to change the caller. But this coupling is visible and is fine. Ideally you still want to minimize the coupling as much as possible because you want to be able to modify the callee without impacting the caller. The type of coupling that is bad though, is hidden coupling. It is when the same design decision is reflected in multiple parts of the code. If you change one part of the code, you will also have to change the other one. But because the coupling is not visible (as the function calling example), you might forget to update the other code and bug happens.

Yehonathan Sharvit18:03:18

Would you say that a frontend and a backend that communicates over HTTP via JSON are tightly coupled or loosely coupled?

benoit21:03:57

From a logical point of view, the coupling is the same as passing a map or a Customer instance. Both sides have to agree on a representation for the message, whether it is a Clojure map, Java instance, or JSON document. But components interact in other ways. By separating the system into a client and server we got rid of a lot of other interactions: propagation of certain kinds of error, they don't rely on shared state, the client can timeout if the server fails to respond... In this sense we reduced the coupling between the two components because we prevented certain kind of change or error to propagate between the two.

andy.fingerhut18:03:35

I'd say that isn't enough information to tell.

andy.fingerhut18:03:55

I'm thinking of an integer. Is it even or odd? You don't know yet.

jkrasnay18:03:07

My takeaway from the Hickey talk (the same one, I think) was that JSON/HTTP services are already loosely coupled, and that pure functions with immutable data seek to replicate that looseness.

andy.fingerhut18:03:28

I don't know if this is a good example or not -- interested to hear from others. Suppose your HTTP server checked the incoming JSON messages from the clients to ensure that they had keys x, y, and z, but gave an error if any others were present. Some would prefer to do this as a form of error checking / validation, but it does mean that as you want to extend the data model over time (assume you do), you need to change that error check at the boundary of the server, in addition to internal implementation changes related to the new data.

andy.fingerhut18:03:28

If the "only has keys in this set" check is in some HTTP handling code far from the core of your application that uses those keys, then there is at least some level of coupling between the error checking part of the code, and the other parts that actually use it. If you have 10 software components between the network interface and the part of the HTTP server that uses the data, and they all do closed world assumption checks that reject keys outside of a valid set, then all of those must be changed whenever that set grows.

benoit21:03:44

For me that's an example of hidden coupling. The design decision (what is the set of required keys) is replicated in multiple places in the code. Every time you change one part, you will have to change the other parts as well. We should definitely not do that 🙂

cgrand22:03:26

with a map (or an http message) it’s easy to have a custom header. It’s safe because things are namespaced (ok X-MyHeader is not that great) and because the thing is built on “must-ignore”. You can have many intermediaries and still get your custom header in the end. Would all intermediaries have modeled and stored only the finite set they know/care when they were designed you wouldn’t get your custom header.

👍 3
benoit22:03:15

By reusing an abstraction you can extend it and implement features that you can reuse everywhere. You can also augment your map with metadata… This is the leverage you get from using a uniform data structure/abstraction. I would not call this flexibility “loose coupling” though.

cgrand08:03:37

I don’t believe there’s such a thing as loose coupling. Coupling is a continuity and describes how much a local change is going to ripple through dependencies. It also affects deployments as strong coupling is at odds with forward/backward compatibility. When I got started in Clojure, it struck me that most defaults were the opposite of what you get in Java (immutability, typing, collection-based modeling, pervasive namespacing on fields). In my experience it’s easier to get a less coupled system in Clojure than in Java because of that. In Clojure you go with the grain of the language, in Java you go against.

Yehonathan Sharvit14:03:48

I don’t think that coupling is only about how changes ripple through dependencies. According to https://en.wikipedia.org/wiki/Loose_coupling: > In https://en.wikipedia.org/wiki/Computing and https://en.wikipedia.org/wiki/Systems_design a loosely coupled system is one in which each of its https://en.wikipedia.org/wiki/Software_component#Component_Definition has, or makes use of, little or no knowledge of the definitions of other separate components. It’s also about the amount of knowledge need to know about each other. A fronted and a backend that communicate via JSON are loosely coupled as the only knowledge that they have about each other is the data format and the semantics of the fields. However, two components inside a single program that communicate by calling function that receive an instance of data class are more tightly coupled). I agree that coupling is a continuum. In the context of function calls, I see three levels from loosely coupled to tightly coupled: 1. argument is generic data (e.g. map) 2. argument is a data interface 3. argument is a concrete data class Does it make sense to you guys @me1740 @cgrand? Once we agree about the definition of loose coupling, we can start discussing about the benefits of loose coupling inside a program.

benoit15:03:34

For the discussion, let's agree that coupling is about the amount of knowledge pieces of code need to know about each other. I think it is related to saying that changes propagate because if A needs to know something about B then when this something in B changes, A needs to change as well. So I don't think we're too far off in our definitions 🙂 Now, if you want to be precise, you need to explain what it is that you don't need to know in case of maps or data interface. Or at least give an example of knowledge that is required in one case and not the other.

benoit16:03:34

Another way to think about it. In your Java version, if you passed an instance of a Map with the customer data as key/value pairs. Would you say that it is comparable to the Clojure map in terms of coupling or different?

cgrand17:03:55

Mostly, with two differences (most signgificative first): The immutability brings you peace of mind that the map isn’t going to change under your feet (which an immutable facade doesn’t bring you). It’s temporal decoupling: you can stash the map for later use without having to care about the object lifecycle (eg pooled/reused object or mutable event in UI). Namespaced keys (that you can emulate in Java).

benoit18:03:11

Yes, definitely, the Clojure map has many advantages. It is also not a type casting nightmare to get a value out of the map 🙂

Yehonathan Sharvit04:03:48

I did a research about the term concretion and it comes from the dependency inversion principle (the D of SOLID) from OOP: https://en.wikipedia.org/wiki/Dependency_inversion_principle: Entities must depend on abstractions, not on concretions. It means that when one class depends in an instance of another class, it should not depend on concrete instances of the other class. Rather, it should depend on an abstract interface implemented by that class. We could take it one step further and claim that a more abstract way is to send a message to an object. The only required knowledge is the name of the message. And it leads us to Java OOP vs. SmallTalk OOP (message passing style).

Yehonathan Sharvit05:03:57

In a sense “just use maps” is an application of DIP to data: instead of passing instances of a data class or of an abstract data class, we pass generic data represented by immutable maps.

Yehonathan Sharvit05:03:02

We have a ladder of abstraction in code and data: 1. Code: concrete class, Data: concrete record 2. Code: abstract class, Data: abstract class with getters 3. Code: message passing. Data: generic maps

Yehonathan Sharvit05:03:35

In a nutshell, one could say that: > “just use maps” is to data what “message passing” is to code

3
Yehonathan Sharvit05:03:14

That’s where Alan Kay and Rich Hickey meet.

cgrand08:03:37

I don’t believe there’s such a thing as loose coupling. Coupling is a continuity and describes how much a local change is going to ripple through dependencies. It also affects deployments as strong coupling is at odds with forward/backward compatibility. When I got started in Clojure, it struck me that most defaults were the opposite of what you get in Java (immutability, typing, collection-based modeling, pervasive namespacing on fields). In my experience it’s easier to get a less coupled system in Clojure than in Java because of that. In Clojure you go with the grain of the language, in Java you go against.

Yehonathan Sharvit14:03:48

I don’t think that coupling is only about how changes ripple through dependencies. According to https://en.wikipedia.org/wiki/Loose_coupling: > In https://en.wikipedia.org/wiki/Computing and https://en.wikipedia.org/wiki/Systems_design a loosely coupled system is one in which each of its https://en.wikipedia.org/wiki/Software_component#Component_Definition has, or makes use of, little or no knowledge of the definitions of other separate components. It’s also about the amount of knowledge need to know about each other. A fronted and a backend that communicate via JSON are loosely coupled as the only knowledge that they have about each other is the data format and the semantics of the fields. However, two components inside a single program that communicate by calling function that receive an instance of data class are more tightly coupled). I agree that coupling is a continuum. In the context of function calls, I see three levels from loosely coupled to tightly coupled: 1. argument is generic data (e.g. map) 2. argument is a data interface 3. argument is a concrete data class Does it make sense to you guys @me1740 @cgrand? Once we agree about the definition of loose coupling, we can start discussing about the benefits of loose coupling inside a program.

Yehonathan Sharvit04:03:48

I did a research about the term concretion and it comes from the dependency inversion principle (the D of SOLID) from OOP: https://en.wikipedia.org/wiki/Dependency_inversion_principle: Entities must depend on abstractions, not on concretions. It means that when one class depends in an instance of another class, it should not depend on concrete instances of the other class. Rather, it should depend on an abstract interface implemented by that class. We could take it one step further and claim that a more abstract way is to send a message to an object. The only required knowledge is the name of the message. And it leads us to Java OOP vs. SmallTalk OOP (message passing style).