I, personally, wouldn’t try to get around it, and just learn how they do it at the company, learn the style (even if only as a curiosity) and experience the tradeoffs yourself so you know why FP does FP things. Less conflicts, more perspective.

how does one even do FP in golang

https://twitter.com/vincentdnl/status/1291041278264713220 https://pbs.twimg.com/media/EelADOtWkAEVZ6o?format=png&name=4096x4096

So there’s this quote in the “History of Clojure” paper: > Type errors, pattern matching errors and refactoring tools are venerated for facilitating change instead of being recognized as underscoring (and perhaps fostering) the coupling and brittleness in a system. I’d be interested in hearing more on this topic. I’ve seen the “Maybe Not” talk and I appreciate how nullability being tied to the type of a thing rather than the context where it’s used can be problematic, but I don’t quite see how type errors or refactoring tools, for example, would be conducive to fragility, as such. An example would be particularly enlightening. Any thoughts? (I’m not interested in pitting static and dynamic typing against one another. I’m just interested in the reasoning behind the quote.)

my take on it is that people falsely venerate the idea of changing something and having the compiler tell you the 57 places that need to be fixed

that's a signal that there are 50+ things that shouldn't care, but do

"but the compiler helps me with the coupling" is tool-assisted coupling

the problem is coupling

I see, that does makes sense. Thanks!

That's a lot more eloquent than the long reply I was still halfway through writing but... yeah, what he said!

I view it as a bit similar to the way folks seem happy to accept vast amounts of boilerplate in Java because "my IDE writes that code for me".

The people from "strongly typed" (whatever this may mean) languages will tell you that their language is a whole different experience from Java, but you still have the same coupling problems I think.

This is all contributing to why I like really simple tooling in my editor and around my language: I don't want to end up depending on "magic" happening and lose my connection with what's really happening under the hood.

(`lein ring ...` is a good example of that, in my opinion: beginners who start out that way don't know how to start the HTTP server themselves and so they don't adopt a "healthy" REPL workflow around working with server processes)

Type systems aren't so much magic though. Garbage collection, JIT, etc, is more magic than types I would say.

Code generation of getters/setters, code modification through refactoring tools...

there are a lot of things that can shore up bad designs, static type checks can do that, editors with "jump to source" shortcuts facilitate bad code organization / factoring

The same could be said of garbage collectors and functional programming: they facilitate writing slower code :)

there are multiple metrics of good vs. bad code, legibility is the only one that is fungible, it can literally be traded in to gain other improvements

if gc and fp help me improve legibility, they can free my time to improve performance or resource usage where it matters

if I don't have legibility, I have nothing

what I don't like are language features / tooling features that help people skate further on reduced legibility

Some people would argue that types help them read code better, since you don't have to guess what f and xs are.

but they use type inference so they aren't even able to read the type in their code

it's something that informs the tool, they then leverage that tool

I'm not saying "no types", I'm saying they are over emphasized

us advocates of dynamic typing generally assume that without types, people will structure their code in a way that is less coupled and better organized

advocates of static typing generally assume that people will be able to encode useful properties of their system into their types, and that doing this is helping solve some business problem

IME both of these are wrong lol

the only times that typing problems have been a serious issue in my clojure projects have been when I'm collaborating with people who are trying to code the way they would in ml, without any of the tooling that makes this productive

@noisesmith can you give an example of that?

@borkdude refactoring the unabstracted implementation of a concrete data type touched by multiple microservices and marshalled via a shared io library

in a language with strong types, this is trivial, though tedious

in clojure, it's a huge waste of time

root problem: you didn't abstract the data properly in the first place

which you can get away with in ml where the compiler takes care of it for you

weeks of time wasted on adding a few bytes to a field

@noisesmith I wonder what they did to marshall that thing to e.g. JSON? In Clojure we do (cheshire/generate-string ...), in typed languages you often describe a per-field mapping to JSON, which is tedious (but can also yield much more performant code if you don't want to serialize everything). How did they do that in Clojure?

this was an overly coupled combination of the apache avro lib (used with kafka) tied to clojure.spec and some overly clever glue code

right

diagnostically, it's the failure to create an explicit boundary between "data from the outside" and "data inside my application"

this was also enabled / festered by using a monorepo, so each service could exploit internals of others

I'm not sure if monorepo has anything to do with it. I once worked in a Scala team where they had many repos and one of the repos was called types :P

anyway - half assed imitation of strong typing ended up with the worst of clojure and ml in one sweeping endless-layers-of-slowly-exposed-bugs refactor

@noisesmith Do you think just throwing EDN on Kafka and use spec on that EDN would have been better?

sure - monorepo didn't cause the problem, attempted typing didn't cause the problem, clojure's laxity with data didn't cause the problem, but they all contributed to the storm that occurred

@borkdude I think it would have been more tractable, but this is more about the way that contracts between services are managed, and how they are designed in order to evolve together IMHO

Once again proving that the really difficult bits of software development have nothing to do with things like typing. 😛

also there was an attempt to make spec generate avro consumers/producers and this was exciting but in the end a source of a lot of problems, it would have been less work and less brittleness to just have the two definitions, or a DSL that was smart enough to create a spec and avro schema that agreed with one another

I heard Mia (https://twitter.com/buttpraxis) talk on defnpodcast about her work with Kafka and spec. She said that they would almost never write specs directly but write code generating the specs for them. Do you perhaps work at the same company? ;)

we were on the same team

I think my memory / long term experience with the avro generator is less happy than hers

Ah.

then again, thinking out loud at this point, if we were using a strongly typed language, we probably wouldn't have tried to pretend that our message exchange format was the same abstraction as our data types / language level type verification

I mean, that would be absurd

but if engineers on our team didn't think that data types were the solution to contracts between services, that also would have prevented the problem

uncanny valley, worst of both worlds IMHO

but I wouldn't characterize the company, our team, or those collaborators by this situation, it's just the worst thing technically that came up, IMHO

does the team still exist?

oh no this was years ago, and many people left shortly after the service went live

I guess this is one of those things where you can write tons of books "Patterns of Enterprise Blah architecture" about

Zach Tellman (who consulted with us about architecture), was writing Elements of Clojure (a very good book about how to do architecture correctly) at the same time he consulted with our team

in fact if people had followed his advice more carefully we wouldn't have had this issue

I confess I have that book but still haven't read it

One counterargument I believe I’ve heard is that in some projects, the realities of software development can drive teams into situations where you just need to make a change that requires a fix in 57 places, and if you have static typing, you at least have some help in making that happen. Yes, the real problem lies elsewhere, but whatcha gonna do? Something along those lines.

right - it's a strategic enhancing of brittleness to make things fail fast so you know they are fixed

Exactly.

but it mixes very poorly with a strategic flexibility that lets you not care about the implementation details of other parts of the code

unless the relationship between the two is very well defined

Yeah, I can see how compound interest can start accumulating there.

it's as if a tent maker and a skyscraper maker were trying to make modular parts of one structure

Maybe a campsite on a skyscraper would work

sure, that's a very well defined relationship

on a human level, it turns so easily into a type-advocate bitter about the sloppiness of the language making these problems hard, and a dynamism advocate bitter that coupling is making the problems hard, and hey they are both right - the mismatch and ill defined boundaries are the problem, not the things on each side

which is what I like about what @lilactown said above https://clojurians.slack.com/archives/C03RZGPG3/p1596739137163400

so no matter what language you throw at it, the system architecture is the more important issue. which is also what Rich Hickey has emphasized in his talks I think

that's my take, but hey I'm a clojurist :D

imagine how terrible ring would be, if the core data abstraction inside your middleware and handler code was the TCP packet

we recognize how wrong that is, but do the equivalent too often in microservices

what would be a microservice equivalent of what TCP is to ring?

whatever data abstraction describes your IPC - so the avro message, or json, or transit, or edn - I'm advocating for having an explicit boundary between app data and data sent to other services

unless you control all those services, it's the same problem with internal APIs and public or library boundary APIs?

people learned the lesson earlier with XML / CORBA but less generally than they should have I think

e.g. don't return field :foobar/id unless it's important. you can never remove it, since that would be breaking

@borkdude yeah actually, that's a fair point as well, though within some inner domains it makes sense to keep one data format for sanity's sake instead of having onion layers of transforms

the process itself is a nice place to have the explicit boundary

This is why I advocate for three separate representations, if only in naming: API (external) data, Domain (internal) data, and Persistence (external) data -- what you expose to clients, what you traffic in within your business logic, and what you write out/read in from constrained external sources.

That said, in many cases, you can control enough of the Persistence format to let it and your Domain format overlap substantially.

another classic example of doing this very wrong: ORM

same basic problem at root - trying to impose coupling across a boundary that's better left standing

ORMs are "wrong" on so many different levels...

the O level, and the M level mostly :)

(one of the inherent problems is conflating identity and state there, between objects and relations)

I find it interesting that I've felt smug about ORMs for a while, being a functional programmer, but fell into the same mistakes on microservice design

the problems can persist even with immutable data as values, if you don't have the right overall structure

We're lazy programmers. We look at the two domains and we squint a bit and we convince ourselves they're "close enough" and we don't need to bother designing and building an actual "mapping" to separate them.

I've just given up and assume anything I do where I feel clever about it was a bad idea

I might still do it

But I'm waiting for the other shoe to drop