Fork me on GitHub
#other-languages
<
2021-03-06
>
Yehonathan Sharvit19:03:20

A few thoughts and questions related to the value of “loose-coupling” between code and data: In Clojure, a `customer` namespace that manipulates a map that represents `customer` data is loosely coupled with customer data. While in Java, `Customer` class with a static methods  that manipulate a `Customer` record (or an immutable data class) is strongly coupled with Customer.  One benefit of Clojure approach is that the namespace doesn’t have to “import” the class definition for Customer On the other hand, a Java developer might argue that the contract between code that manipulate Customer data and Customer data shape is not explicit  For instance, if you want to rename a field in Customer data, how would you discover all the pieced of code that need to be updated Rich likes to say: > We should program the insides of our sytems like we progam the outsides A Java developer might object: >  Does it worth it to have code and data loosely coupled when they both live in the same program?

benoit02:03:52

I'm not sure to see the difference in coupling in your example between Clojure and Java. For both languages, the function that manipulates the customer is coupled to the description of the customer. If you change the key in your Clojure map representing customer data or if you change the name of the attribute on your Java class, both functions will not work as expected. For me, that's what coupling means. A change in one part propagates to the other part. Two things are coupled because they are somehow (logically) linked together and effects propagate over this link.

Yehonathan Sharvit01:03:52

In the data-oriented programming approach (the one Rich taught us), code and data are coupled in the same sense that a frontend and a backed are coupled via an API definition. If the backend breaks the API definition, the fronted code has to be updated. But they are coupled in a loose way. Meaning that one doesn’t care about the implementation details of the other

benoit12:03:41

> "one doesn’t care about the implementation details of the other" This is abstraction, the distinction between an interface and an implementation, and both, Clojure and Java, support it. The functions in Java and Clojure that manipulate those records and maps are coupled to the definitions of these records and maps. > One benefit of Clojure approach is that the namespace doesn’t have to “import” the class definition for Customer It is not because you don't have to import the class customer that it is not coupled. In Java, the coupling is visible because, like you say, you have to "import" the Customer class. And the functions that take a customer as parameter specify the type of the argument. In Clojure, you don't have to "import" the Customer class. This is actually a drawback of because in doing so you create a hidden coupling. Specs are one way to make this coupling visible. Coupling is a concept at the logical layer of software, not at the concrete implementation layer. For a definition of these layers, see http://www.pathsensitive.com/2018/01/the-three-levels-of-software-why-code.html Don't get me wrong. It is much better to represent records as maps rather than Java classes, but it has nothing to do with abstraction or coupling.

Yehonathan Sharvit15:03:10

@U3E46Q1DG Could you rescue me here? I couldn’t find the proper words to answer @U963A21SL’s questions 😬

Yehonathan Sharvit15:03:16

@U3E46Q1DG You are so convincing with your sophisticated vocabulary (strawman argument, discrete getter, typed generic getter, payload evolution, external state, concretions …). I wish my french was as refined as your english 😊

Yehonathan Sharvit15:03:23

@U963A21SL the connection between loose coupling and representing data with maps was made by Rich Hickey on his talk https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureMadeSimple.md (Around 0:10:29).

benoit16:03:30

I'm not seeing the connection between loose-coupling and using maps. He says that we need flexibility and loose-coupling, I think we all agree on this. Then there are some quotes. Then he talks about the fact that we should recognize that what we manipulate most of the time in our programs are records, simple data, and that we don't need fancy abstractions on top of it. I agree with this. When your program manipulate records of data then you need to reuse the same abstraction for all your records (like a Clojure map). Sometimes though, it will help you to create new abstractions to control the complexity in your programs. You have to find the right balance between reusing existing abstractions and creating a new one. You can do the same in Java by using the Map abstraction to represent your records instead of creating the Customer class but it is much more awkward to use, especially with static typing. Regarding coupling, wether you have customer.getName() or (:name customer), your function in Java and Clojure will be coupled to the interface of your customer. There is just no way around it.

cgrand16:03:34

Couple of remarks (since @U0L91U7A8 summoned me): keywords namespacing is important and one keyword could be seen as 1 interface. You couple against what you use vs coupling against the whole thing.

benoit17:03:14

Agree in this case. That has an impact on coupling. The corresponding operation in Java would be to put the subset of method(s) in an interface and have the function depends on this interface rather than the class.

Yehonathan Sharvit18:03:56

Accessing data in a map via a keyword is fundamentally different than accessing data in an object via a method (or a member). The only information that needs to be known in order to access data via keyword is the name of the field. That’s how I understand the loose coupling. Of course there is a coupling. The question is: is the coupling loose or tight? I’d like to claim that accessing data in a map is loosely coupled with the representation of the data while accessing data in an object is tightly coupled with the representation of the data.

Yehonathan Sharvit18:03:32

But I cannot find the proper arguments. I’m not even sure that I formulated my claim properly.

Yehonathan Sharvit18:03:56

But I’m quite sure that when Rich talks about loose coupling, he refers to “just use maps”

benoit18:03:35

I would suggest that you define the terms before making a claim. To make sure you say what you mean and to make sure people don't misunderstand you. Coupling is about propagation of change. Two things are coupled if changing something in one thing requires changing something else in the other thing. The change propagates from one to the other. Two things will be tightly coupled if every time you change the slightest thing, you have to change the other thing as well. And two things are loosely-coupled if you have a lot of degrees of freedom when changing one thing without impacting the other thing. A lot of software design is about finding the right level of coupling to make sure that things work together (there will be a mimimal amount of coupling) while still being able to change each thing independently (without having to change any of the other things).

Yehonathan Sharvit19:03:38

I am not sure that your definition of loose coupling is accurate, at least that’s not what Rich Hickey means by loose coupling. Richen often claims “we should program the insides of our systems like we progam the outsides” The interfaces between two components inside a program should be the same kind of interfaces between two programs. When two programs agree on an API, each one of them is free to change the implementation providing that they don’t break the API. I’m pretty sure that this is what Rich means by loose coupling. Now, how the API between program should be formulated ? In terms of data. Not in terms of code (like DCOM etc…) According to Rich’s philosophy, we components inside a program should define their API in terms of data. Not in terms of classes and methods.

benoit19:03:53

> When two programs agree on an API, each one of them is free to change the implementation providing that they don’t break the API. I’m pretty sure that this is what Rich means by loose coupling. That's what I also mean by loose coupling. The changes you make to your implementation do not propagate to the user of your API as long as you respect the contract of this API. By "define an API in terms of data", do you mean making sure that the functions of your API take data as input and return data as output? If so, it might make sense for software modules that deal with information records but there are APIs that do other things that simply manipulating information records. When Rich says "use maps", he means that when we want to represent records of information, we should use maps. And I totally agree with that. But it doesn't mean that we should use maps of static data everywhere. Take an API like Datomic that Rich designed himself. Would you call the database value that you pass as argument to most of its functions data? Thankfully you don't have a gigantic map containing all the data in your database. The API is a set of functions that lets you query this database value.

Yehonathan Sharvit02:03:58

I think we agree on most of the things

andy.fingerhut19:03:10

There are times when Clojure developers want to use Spec, but want the "closed world assumption", e.g. if they didn't define a key in a map explicitly, it should be an error for it to be there. Rich has argued that very often when someone does this, they later end up regretting it as they wish to extend their data model.

borkdude19:03:15

> For instance, if you want to rename a field in Customer data, how would you discover all the pieced of code that need to be updated Namespaced keywords and something like clojure-lsp or Cursive ;)

Yehonathan Sharvit21:03:27

Interesting @U04V15CAJ It means that decomplected keywords are more intrinsic to data-oriented programming that it may seem at first!

borkdude21:03:28

I think that's one of the main ideas about spec, inspired by RDF: each attribute name indicates something unique

borkdude21:03:42

so :customer/name always means exactly that

Yehonathan Sharvit21:03:46

It means that the loose coupling between code and data is in fact not between code and maps but between code and map fields.

borkdude21:03:03

Yes, that's also one of the core ideas: you can have arbitrary combinations of fields, there is not such a thing as a type which is a fixed combination of fields

borkdude21:03:33

Watch Hickey's talk titled "Effective Programs" or "Maybe Not" (I think the last one emphasizes this idea more)

Yehonathan Sharvit21:03:53

I think I watched those talks

Yehonathan Sharvit21:03:11

I understand the idea behind decomplected keywords

Yehonathan Sharvit21:03:37

I always thought that Clojure rationale was: “just use maps” because maps are more appropriate than classes to represent data to be manipulated. Now, I understand that according to Clojure, “there is not such a thing as a type which is a fixed combination of fields” as you wrote. In a sense, it means that according to data-oriented programming, field keys are first-class citizens.

borkdude21:03:52

yes, mixing attributes that travel together

borkdude21:03:18

and this "together" is usually maps

andy.fingerhut19:03:01

There are uses for the closed world assumption, I think, e.g .when you are using experimental early code versions to learn what the data model is, iteratively, e.g. you add a few keys for a map that you know should be there, but you want running the code to tell you if any other keys are found you haven't anticipated, with an exception or similar error.

andy.fingerhut19:03:38

Later, after you believe you have done enough experiments that you have covered all of the keys that exist today, you switch the spec to allow other keys in the future, if you believe that the open world assumption is the long term way to enable growth of the data model.

andy.fingerhut19:03:17

But lots of programmers (myself included) feel better knowing they they have covered all of the existing ground of the data model, even if there is no explicit documentation for what the data model is, so experiments on large data sets are the best we know how to get there.

andy.fingerhut19:03:17

Lots of developers also like it if they have errors occur in a closed-world assumption data model forever in their code, because they believe that will let them know when the data model (and assumptions they may have built into their code based upon it) have changed, in some other piece of software that someone else has modified, without informing the world.

andy.fingerhut19:03:15

That liking of "I get errors if the data model is extended by someone else" is not particular to developers of one programming language, I don't think. It is a general feeling of "I want a loud noisy error message if the world has changed around me".

andy.fingerhut20:03:45

Java classes with a fixed set of fields, and similar constructs in many other languages with similar kinds of type systems, enable that closed world assumption by default, but in most of those language also tie the kinds of functions / methods / code that can update and read those fields to the specific class or type name. Clojure maps don't, but leave many developers with that uneasy "I might have forgotten to change all occurrences of the code that should be changed when I add a new field/key-in-my-map" when certain kinds of code changes occur.

andy.fingerhut20:03:25

Probably a way that Rich Hickey might describe that is (my words here, NOT his): "many developers like that form of coupling. They have become accustomed to it so much that they rely upon it, and cannot imagine developing software without it".

Yehonathan Sharvit21:03:24

Could you guys think of other benefits of having a loose coupling between code and data?

Eccentric J02:03:29

I’ll give it a shot 😅 So at a former job, we had this very specific Bid class that represented a Bid model, stored in a bids table in the db. It had methods for dealing with it from all possible contexts: public users, vendors, and admins. The problem is, the idea of a Bid is actually pretty different between each group and by the time I left, the Bid model was gigantic with methods like .send_emails then .send_vendor_emails and .send_user_acknowledge_emails. If this were Clojure we could have had a common set of attributes that represented all possible fields from the db, then defined a Bid for each context separately by composing those attributes and having groups of functions for operating on it for each context like company.user/send-emails and company.vendors/send-emails. Subclassing addresses some of the surface level problems but in reality it’s the same data type, but have different meaning in each context. I hope that makes sense.

Yehonathan Sharvit08:03:35

@U8WFYMFRU I think your example illustrates the benefits of separating code from data. Could you have gained the benefits you mentioned by storing bid data in various data classes like BidForVendors, BidForUsers …?

Eccentric J16:03:23

If you know all the possible contexts from the beginning, it may be possible to get some of those advantages with subclasses. But realistically, new contexts emerge which means a continuous cycle of moving methods and attributes from the base class into the subclasses. The one wall you might hit is if you want to perform a sequence of actions across contexts. So if an admin closes the bid, we may want to send closed emails to both the vendors and users. With that separate Clojure setup, that’s trivial to do since you have a single bid object hash-map that can be passed between functions across any context. With subclasses, you would need to create new instances of the next subclass you need:

Eccentric J16:03:16

(ns company.admin.bid.api
  (:require
   [company.db :as db]
   [company.user.bid :as user]
   [company.vendor.bid :as vendor]))

(defn close
  [bid]
  (let [closed-bid (assoc bid
                          :closed true
                          :status :closed)]
    (db/update! db/conn bid)
    (user/send-closed-bid-email closed-bid)
    (vendor/send-closed-bid-email closed-bid)))

Yehonathan Sharvit16:03:36

I see. I think it’s called the ClassAWithThisAndThat problem

Yehonathan Sharvit16:03:52

You need to create a class for every combination of fields!

Yehonathan Sharvit16:03:58

That’s insane 😱!

Yehonathan Sharvit16:03:15

What’s even more insane is that Java folks don’t seem to be aware of this insanity