Basically, yeah! At least in spirit. I’ve considered the convention of speccing every function that takes or returns those central objects, as well; as long as it can be done in a non-obtrusive way, it might be desirable. Speccing lots of functions with lots of specs makes for lots of extra code, so that may not be that valuable in terms of readability. But speccing some functions over and over with the same business objects gives you familiarity so that you can glance over it.

And I dig the idea of closed specs. I get that open specs are in the spirit of Clojure, but it leaves open a massive door of doubt. “If I put in extra keys, will that cause a side effect? Should I avoid doing that?” or, conversely; “Someone put in extra keys, and it’s causing a side effect. Is this intended, and therefore missing from the spec?” or: “Someone put in extra keys, but they have no effect. Should they be having an effect? Regardless, should they be added to the spec?” You can have conventions that make the answers to these questions more or less obvious, of course.

Why would an extra key cause a side effect?

Might be far fetched, but I could see it happening. • You’re sending off the data structure to an API that you don’t own • Some code goes through each key-val to do something • other 🙂

Those are typically called "code at the edge of your system" and this is where you can do select-keys or whatever to make sure no garbage is sent out

sure 🙂

You can do the same inside each function that takes a map. “Ah, this map can potentially hold the whole world, but I’m going to select these 3 keys that I’m interested in”. I kind of like delegating that responsibility to the caller; clean up before calling.

Don’t leave in-between mess. Clean boundaries.

I think as long as you're "in the system" you shouldn't care about extra keys, only if you're dealing with "no-clojure-philosophy" systems like foreign APIs or databases

Agree with @U04V15CAJ here. Within the system, you should be ok with extra keys. When shipping stuff out, there might be keys that you don’t want to ship, like tokens/passwords/etc, so it might be nice doing select-keys or set/project if you have a list of things. Also, it might make sense to architect for avoiding forgetting to do select-keys, so that you split domain objects which has sensistive info on it, ie split a user into a user which is benign to ship off and subject which could contain passwords/tokens etc.

How about having an or schema to parse the data and figure out which case it really is?

Could you provide some code to exemplify the issue? As far as I understood it, you are concerned about a possible "mismatch" between the functions and the provided input data. Whenever I see "spec" I think of an xsd schema. I am new to Clojure so please excuse my ignorance 🙂 .

“Not caring” is fine when you’re writing new code. I’ll take a map in my function, destructure the keys I’m interested in, and not care about any remaining keys. Callers can include the world in their map argument; I’m not worried about it. It’s different when you’re maintaining a large code base. I want to refactor a data structure foo to be slightly different for whatever reason. To do this safely, I should check and possibly refactor all usages of foo. How will I know when I’ve found all code paths that make some use of foo? In Clojure, this can be hard, as foo is not always referenced by name (and therefore greppable). There are many ways for code to treat data generically and anonymously. Allowing everyone to include the world in every map that’s sent to every function increases the search space. foo can appear in every call to every such function. Potentially, it can appear many times in the “world” maps, nested into other data. If I can limit the input and output of functions, I limit my search space greatly. Like I said, there are other conventions that can limit your search space, like good code organization (“only these namespaces are concerned with foo“). And I don’t feel certain that speccing every function, or maybe even most functions, is a good solution. But I’d love to see how it feels.

if you use something like clojure-lsp, it's easy to find all usages of foo

And perhaps it’s not that big of a problem. I just work with a large code base with a lot of other people, where I’m often annoyed that I can’t change things with more certainty or quicker. Usually, there’s lots of digging involved, and, sometimes, even after being very thorough, I have to shrug and go “Best I could do, let’s see if any defects roll in”.

@U04V15CAJ uhm… does it compile the code and exhaust all code paths? Or what do you mean?

Wait, not compile the code. Analyze it somehow.

you weren't referring to foo as a function name?

No, a data structure 🙂

I think this discussion has two answers: 1. Use a validation / coercion library at the critical edges (spec, schema, malli) 2. Switch to a statically typed language

Yeah of course 😄

I find myself weighing the pros and cons of static vs dynamic a lot when pondering this stuff. The static languages I’ve experienced were also annoying (mostly Java).

TypeScript is all the rage man ;)

yeah I know 🙂

I’ve been meaning to try it…

I've been in this phase myself. I explored Haskell for a year or so. But I came back to Clojure every time

Perhaps Typed Clojure could also help you

yeah, maybe

I’m asking the impossible, really. I want dynamism, but with the certainty of static typing. opt-in like in Typescript is perhaps the best bet.

Now to get the same in Clojure 🙂

Aspects of the Haskell typing intrigue me, but not enough.

I just listened to The REPL about #jank - Jeaye want to create something like typescript (gradual typing) in his clojure dialect based on a malli-like syntax

Wouldn't Python be a better candidate then Typescript? Isn't refactoring without tests an issue in any code base? Or would tests not be able to catch breaking data structure changes?

Here’s a fun example, which might not be related, but still:

Deep down in our database code I wrote in a function:

(if (sequential? upsert?)
                    upsert?
                    [:_id]))]

Now, who gives us a sequential upsert? I have no clue, I have to search for it, manually.

And the place looks like:

(assoc (mongo/requested-version-ctx ctx requested-versions)
                                                 :pg-opts {:upsert? [:_id postgres-rebaser/branch-column]})

So what I did just now, was to add this fn deep down in our database code:

(defn upsert-on-conflict [ctx cols]
  (update ctx :pg-opts merge {:upsert? cols}))

(postgres/upsert-on-conflict (mongo/requested-version-ctx ctx requested-versions) [:_id postgres-rebaser/branch-column])

I believe the effects of doing stuff like this are very profound.

@U04V5VAUN I think you gave a talk about this kind of usage at ClojureD in 2018 or so right

no 2019

I gave a talk on something around this subject, but I don’t know if this concrete “technique” was in it.

A talk that I would really like to give, but I haven’t figured out yet is “How to write your programs so types become irrelevant”

well, the title would be provoking enough :)

Where “types” are to be understood as static type types. Because we all use (and probs think in) types even though we code in a dynamic language.

The only thing I’ve managed to come up with so far is that we should know very little about data that travels far.

Like, within a ns, say a ns which deals with users, you can have a lot of knowledge about what a user is. And since it’s all very local, you tend to manage to remember what a user is. But, if you send that user far away (via fn calls) down the stack, the recipients should probs just care that it has an id and a name, because, that’s about what you can remember.

takes long drag on joint > The only thing I’ve managed to come up with so far is that we should know very little about data that travels far

That sounds reasonable. Within some context, you have lots of details, but you should avoid shovelling all of those details onto other contexts. Encapsulation, in spirit?

I think one of the points that slipset made in his talk was that you should access map things using functions

@U04V5VAUN I've been thinking about how to use var references as well. I guess it's really a result of the tools supporting "Find usage" so well 🙂

and if you do that, then you can say: this function should never be used from this and that namespace

which would be your encapsulation

It sounds a bit like the "poor man's object" koan

Yes. And high cohesion. If your whole code base have to know every little detail about what a user is, you're in for a wold of pain. Build some langage around the user.

:thumbsup:

@U04V15CAJ your comment about the joint. Priceless! It made my day :)

The venerable master Qc Na was walking with his student, Anton. Hoping to prompt the master into a discussion, Anton said "Master, I have heard that objects are a very good thing - is this true?" Qc Na looked pityingly at his student and replied, "Foolish pupil - objects are merely a poor man's closures."

    Chastised, Anton took his leave from his master and returned to his cell, intent on studying closures. He carefully read the entire "Lambda: The Ultimate..." series of papers and its cousins, and implemented a small Scheme interpreter with a closure-based object system. He learned much, and looked forward to informing his master of his progress.

    On his next walk with Qc Na, Anton attempted to impress his master by saying "Master, I have diligently studied the matter, and now understand that objects are truly a poor man's closures." Qc Na responded by hitting Anton with his stick, saying "When will you learn? Closures are a poor man's object." At that moment, Anton became enlightened.

    - Anton van Straaten

https://twitter.com/borkdude/status/1564626237447933961

@U02AMR8032L dno about Python vs Typescript; I know too little about both. You're right (imo) that static types and unit tests are complementary, not necessarily substitutes, though you could probably test your way out of what a type system provides (apart from editor-time verification), by being very exhaustive 😄

@U04V5VAUN do you do anything in particular to signal the coupling of certain ns'es and datastructures?

No. Just heuristics ie reading/studying the code.

idk if there's something clever you could do, like using a constructor fn to add metadata that links the fn to the datastructure

I'm all for conventions 🙂

if you create "language" around your data, some behaviors will automatically look weird to you like: huh, why does utils.logging depend on user.manager

I've been thinking a bit about constructors lately. Like in your user ns, you could have various fns to construct valid users, rather than just typing out the maps all over the code

I know @U050P0ACR has been advocating getters/setters rather than just using eg :name. I find that a bit too much :)

@U04V5VAUN I recommend watching your own ClojureD talk, I think you've been thinking about that since then already ;)

https://www.youtube.com/watch?v=Tq7r97G4b7Y

I could not find until now something that relates to patterns or principles on how to structure code in an FP context. Do you have any recommendations? Just started watching the video :)

This is the kind of thing we just spoke about:

Wow, I’m smart!

And I agree with myself 🙂

I’m internally consistent 🙂

@U04V15CAJ that is why I asked 🙂 I was thinking about more concise literature analogous to the Gang of Four book.

I'm not aware of anything like that, but I would love to hear about it. There's this funny blog post thingy where someone recreates the patterns of Gang of Four with Clojure - it's mostly just obvious function usage.

Functional Programming is somewhat ill defined, I think? Some people seem to strongly associate Haskell style type systems with functional programming. I mostly just think "a language that focuses on/enables usage of higher order functions", e.g. via map, filter and so on. Maybe there's a more complete definition.

I remember seeing this one a long time ago 🙂. every pattern in OO is a function in FP . https://youtu.be/srQt1NAHYC0?t=228