> I have a question about best practices. I have watched "Simple Made Easy" and "Clojure, Made Simple". Hooray for plain old data with constructs that can arbitrarily nest along with a huge standard library of functions that can work with them! The ghost of Alan Perlis smiles broadly. > > Uh, oh. Here come defrecord and defprotocol. The standard advice I read about them is usually along the lines of: Don't use them, at least not to start. You can always switch to using them later, if things start to get messy. > > For now let's forget about any other considerations outside of modelling and expressing intent. Are there not problem domains where it's a good idea to reach for these things right off the bat? > > Let's say I want to write a program that plays "Connect 4" against a human opponent via monte carlo tree search. I'd naturally want to reach for records and protocols to help model game states, the game tree, etc. And that's even before potentially generalizing the program to play other games that have their own rule sets and gui skins. I think the records and protocols would very much help to keep our responsibilities separated and prevent us from complecting things. > > I can't imagine approaching this starting off with plain old data. I'm not sure if this inability represents a hole in my skill set or not. > > I can certainly envision cases where the standard advice makes a lot of sense, e.g., when data transformations are central to the program's mission. I know almost nothing about web programming (where I come from software doesn't have "ends") so at the risk of embarrassing myself, I can imagine a web server working something along these lines, where an incoming request gets sucked into a map, with interesting bits and pieces corresponding to well-defined keys. Then as processing proceeds, new keys can be added and values can be altered until we're ready to use the transformed map to spit out a response. Ta-da! I am now a back end developer π > > But seriously, is there harm in reaching for records prematurely? I understand you can make a hot mess of things if you use the tools improperly, but other than that I'm not sure what the motivation is for the "standard" advice (assuming I haven't misrepresented it).
The standard advice is "just use maps" - the motivation for that is given in section 3.4.2 of this document: https://dl.acm.org/doi/pdf/10.1145/3386321 It sounds to me like it's time to stop studying though, and just make the connect 4 program; the simplest version does not use records or protocols.
@mbk.racket What's your previous programming background? If you feel the urge to reach for records and protocols "naturally", I'm thinking you're used to an OO approach perhaps?
To me, a "game state" is dataβa map. A "game tree" is also data and also probably just a nested map of data. Most of the class "Gang of Four" Design Patterns end up being "just" higher-order functions in Clojure. So it's data + functions all the way down.
records and protocols are for performance. the problem with using them prematurely is that they are less flexible than the other abstraction mechanisms (functions and maps) and introduce unneeded ceremony
When I was learning, one thing that this advice used to confuse me with is I thought it also meant that we shouldn't be modeling entities. But that's not the case, it's just that you can model entities using plain maps and it's just less complex, and more flexible.
(defn make-car
[& {:keys [model make price color]}]
{:type :car
:model model
:make make
:price price
:color color})
And if you want "methods", just use plain functions:
(defn make-car
[& {:keys [model make price color]}]
{:type :car
:model model
:make make
:price price
:color color})
(defn reduce-price-by
[car percent]
(update car :price #(* % (- 1 (/ percent 100)))))
Some people put this inside it's own namespace like (ns my-app.car) but also many people put them all together in say a (ns my-app.domain) namespace or something separated by some comment section headers.
Now my question would be, why is this not sufficient for what you had in mind?P.S.: And idiomatically, you don't go creating "accessors" like "update-model", "update-price" or like "get-price", "get-make", because since it's just a map you can just do "(:price car)" to "get", and since it's immutable anyways you can just do (assoc car :price new-price) or use update if it's dependent on the current value. So you would only add functions that are doing more complex things and modeling more conceptual operations.
Also for protecting invariants, like validation that sometimes is done in setters and getters, that tends to be more that you validate the whole entity after a series of changes. So you might have a spec for car and then say (s/valid? ::car car) instead of having each "setter" validate that you are setting to something valid.
@seancorfield Re: my background Strictly speaking, I was programming in the functional paradigm well before OO since my first exposure to lisp was running a simple interpreter whose implementation I had painstakingly typed in from the BASIC source code listing (80micro magazine, March 1983 for the record). But as a professional, I started with imperative C and x86 assembly (RTOS firmware stuff) but eventually made my way to the the land of Windows desktop application development. So yes, decades of experience with OO, starting with C++/MFC/ATL through the present day with C# and WPF. I'd say my OO thinking has evolved over the years with lessons learned, many due to the influence of FP. (Yes, I'm the guy mentoring the juniors about the benefits of using static classes to define collections of pure functions as a core, the better to mix and match via delegation to solve those similar-but-different use cases instead of (yikes!) code reuse via inheritance.) Outside of work, I find most aspects of software development endlessly fascinating, but most of all I'm a language nerd. I love playing with languages and studying them to try to understand the ideas behind them and the choices that their designers made. What hard things do they make easy? Do they get a lot of mileage out of a handful of orthogonal building blocks? Do they care about reducing the programmer's cognitive load (Zig) or do they champion TMTOWTDI to a fault (looking askance at you, Perl 6 and the bloated monstrocity modern C++ has become)? So I've toyed to one degree or another with a bunch of stuff, either at home for fun or for my own bespoke tools at work, from Perl to Python to Powershell, FORTH to F#, Babashka to Racket to Janet, etc. I did the MCTS game stuff in Rust with extension points for different games (text only though, no GUI). Clojure is the language I wanted to love, but the timing wasn't right (I remember reading the first edition of the Joy of Clojure at the time. My impression was the setup and tooling were, ahem, vastly suboptimal. Especially for a Windows guy with no Java background.) Now with a mini boom of new clojure dialects in the offing, I'm eager to go back and try again freed from the fetters of the JVM. I'm particularly interested in Jolt these days. And in learning about Clojure best practices from the wisdom of others instead of the school of hard knocks. Sorry for such a self-indulgent answer. This was probably more appropriate for "introduce yourself"
@hhausman Thanks so much for the article. I look forward to digesting it.
@didibus Thank you for your very concrete reply. I completely agree that the map approach would indeed be adequate. I clearly had a mental goof in my original message where I somehow equated using plain old data with having a single amalgamated nested data structure like in my web server example. D'oh!
FWIW, I will mention that at least one well regarded source apparently disagrees with the conventional wisdom: In "Clojure Applied", Ben Vandgrift and Alex Miller introduce records specifically in contrast to maps as a preferred way of modelling domain entities. And the way I read it, ISTM this is not just for performance benefits, but for more clearly expressing intent to other readers of the code.
that book came out before the core.schema namespace (iirc Alex was even on the team that implemented schema). I hope I will be corrected if this is wrong, but my understanding was that one of the goals of core.schema was to separate the aspect of communicating design intent from the implementation details. there's a good talk from Rich where he talks about the rationale and references RDF as an influence
Yes, I think records initially were for that, and before them structs. But the idioms evolved where most people eventually realized as I showed that really you don't need a special construct, maps + destructuring easily fulfills that contract. It's not universal and some people like records, and they are usable as maps as they implement the map interface. But they come with a baggage of generating a Java class under the hood (in other dialects this might not be the case). That means records are meant to be: 1. A definition of the shape of a record with a name/doc to describe your application entities 2. A way to create an efficient map-like structure that has faster field access than a map (no hash lookup instead its direct field access) 3. A way to leverage the JDK host first parameter type polymorphic dispatch which is faster than Clojure's multiple dispatch through multi-methods, by instead being able to implement interfaces or extend protocols to them Some things to know: 1. Dissoc will convert it back into a map and return a map of the record minus the removed key 2. Assoc of additional keys (not defined in the record) will be stored in an internal map on the record, so no longer as fast as field access 3. Assoc/update of keys you defined return a copy of the record object with the fields updated. 4. Meaning records are still immutable, but don't do structural sharing like persistent map does, instead it does a shallow reference copy. So why the idiom evolved to prefer maps then? 1. As you see, it's easy to accidentally start with a record and get a map back, you think you can just filter, merge, and so on as if they were a map because it supports it, but as soon as you do that you get back a map. If then you expect that type dispatch still works you'll be surprised it doesn't because you now have a map and the record is no longer of it's original say Car type. 2. They're not data, so serialization and deserialization brings all the complexities of object serialization. 3. Some other little details like that I'm forgetting This is why people now suggest to only use them if the performance gains from direct field access over map key lookups or for JDK based type first arg type dispatch really brings speedup to your use-case. Remark you can have maps implement protocols as well, so you can still define protocols for maps. Protocols are not frowned upon, when you need an abstraction of multiple functions working together protocols are great. That said, oftentimes just functions can work over many maps making them generic, or multi-methods can also be used to branch a function's implementation for different map shapes. Lastly, that leaves one thing about records that's nice, the defrecord is a convenient way to define the shape and name/doc of your entities. For that Clojure has evolved to using Specs such as clojure.spec or some prefer some 3rd party libs that are of the same idea. The spec is your definition with name of your entities. Defrecord is still nice in that it will also create a constructor conveniently for you. But that can be your own macro you make or really it's not that much trouble to write your own make-foo constructor to construct a map of your specs. If you have any questions about any of that feel free to ask. Given your experience I thought I could get more technical and detailed right away.
Oh, and I'd say, even Specs, generally people tend to add them later, because a map constructor function with destructuring is often legible enough in showing you the name + doc + shape of your entities. Specs add value once you want to be a lot more detailed, like specifying the type and allowed values of each key, and include validation of more complex invariants.
@noisesmith I think you meant to say clojure.spec and not core.schema ?
you are correct
Also, plain map can now participate in protocols via metadata, pretty huge.
> Remark you can have maps implement protocols as well, so you can still define protocols for maps. Protocols are not frowned upon, when you need an abstraction of multiple functions working together protocols are great. That said, oftentimes just functions can work over many maps making them generic, or multi-methods can also be used to branch a function's implementation for different map shapes. As a hack comedian once said: That's gold Jerry! Gold!
Currently going back to basics and walking through https://caveman.mccue.dev/tutorial/clojure/16_set_up_database_migrations step by step. Pretty cool. But I am stuck at https://caveman.mccue.dev/tutorial/clojure/16_set_up_database_migrations - how do I download the postgres driver from Maven?π
Quoting the text at that link: > You can find the latest driver https://central.sonatype.com/artifact/org.postgresql/postgresql/versions. The page is painfully slow to load on my end though...
Ah, finally - here's a direct link to the JAR: https://repo1.maven.org/maven2/org/postgresql/postgresql/42.7.13/postgresql-42.7.13.jar You can find it if you first go to the Maven page of the driver, then click Browse to the right of a specific version.
Cool, thanks :) i am Kind of glad that it wasnt the clojure part that made feel stupid in this tutorial:smile: