This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
Hello! I’m new to the Clojure community and I’m just learning the ropes of this new langage. Very thankful for the vibrant community that I’ve found here in this slack. I also bought Clojure for the Brave and True, starting to watch lots of tutorials, the whole nine yards. One thing I was wondering is because of the dynamic nature of Clojure and lack of types, how does Clojure scale to larger code bases? I know you can use spec to define the shape of data that bits of data should be, but is spec normally used? I work at a company with a sizable TypeScript codebase (100+ lines of code total), and adding in types has been an absolute GOD SEND. The amount of times where we thought something was a string but it was a number is immeasurable. When refactoring I feel way safer and our code is far more resilient with static types. So I was wondering a couple of things: 1. Are there any good clojure articles/videos related to why Rich Hickey decided to not build types into the language? 2. Is spec really commonly used and is the “norm”? 3. What are some substantial clojure codebases, and how do they handle safely refactoring without unintended breakages? Any insight would be greatly appreciated!
Hi Gabe! Welcome! Where I work, we have 140K+ lines of code and it's fine, to be honest. Because you work in the REPL with small pieces of code and all data is immutable, a lot of the problems that occur in mainstream languages that seem to require types to mitigate just don't crop up anywhere near as much. Rich has talked about type systems in a few of history conference talks and I expect he talked about it in the history of Clojure paper as well. Spec is used on the boundaries of systems but it is not a type system and should not be used as such. I talk about the various ways we use Spec in this blog post from a few years ago: https://corfield.org/blog/2019/09/13/using-spec/ I don't know the answer to 3 -- I think Metabase is one of the few open source application codebases? Most Clojure libraries are small and focused on solving a single problem, so they tend to be fairly small. As to refactoring, I think the fact that Clojure code generally relies on just a handful of types means that there's a lot less refactoring at that level: maps, sets, and vectors are mostly what we work with, and while the shapes of those data structures may evolve, they don't typically change to other types as often -- and with no custom types to deal with, you don't have such a rigid set of function signatures. For example, if you have a call chain where only the bottom-most function cares about the shape of the data, all of the intermediate functions do not need to be touched if the top-level caller has to pass something different in -- only code that actually constructs or deconstructs the data structure needs to be modified.
This is Rich's "History of Clojure" paper: https://dl.acm.org/doi/pdf/10.1145/3386321 and here's a playlist of his talks: https://www.youtube.com/playlist?list=PLZdCLR02grLrEwKaZv-5QbUzK0zGKOOcr -- I think Effective Programs and Spec-ulation touch on his thoughts about types? There's also a repo full of transcripts which might be easier to search: https://github.com/matthiasn/talk-transcripts/tree/master/Hickey_Rich
Thank you for your response! Appreciate the detail. I’ll take a look at those resources 👍:skin-tone-3:
I think if you're interested in Rich's motivations, whether type systems or something else, one of the best places to look is his pair of talks https://www.youtube.com/watch?v=cPNkH-7PRTk and https://www.youtube.com/watch?v=P76Vbsk_3J0. They mention a ton of design choices in passing. Those talks make it clear that dynamic typing was a central part of Clojure's value proposition from the very beginning: > [Clojure] sort of stands on four points. It is dynamic. It is functional. It is hosted on the JVM, and it embraces the JVM. Searching the https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureForJavaProgrammers.md of the for-Java-folks talk for "dynamic" brings you to extensive discussion of his reasoning:
Why use a dynamic language?
+ Flexibility
+ Interactivity
+ Concision
+ Exploration
+ Focus on your problem
The for-Lispers talk (https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureIntroForLispProgrammers.md) touches on another fundamental motivation, "abstraction and building algorithms on top of abstractions instead of concrete data structures". Clojure is far more concerned with interfaces than types — what a thing can do rather than what it is. I really like how this talk shows how Rich made this leap as part of trying to make a better Common Lisp.
Rich circles back to this point in the context of multimethods but the idea is central to the whole language:
> I do not care what type it is. I do not care how many other attributes it has, or anything. I do not care. I should not care, and I should not dictate to you that it be this particular type in order to interact with my API. That has to stop. Let people just glom stuff on so they can talk to you, and not have to be something so they can interact with you.
I think the answer to your other questions might best be answered by drilling down on Sean's point that spec is not a type system. Zooming out just a bit, Rich's idea (IMHO) is that a dynamic language is better served by a contract system.+1 on “contract system”, and “use spec at the boundaries”. That doesn’t mean you only have one system boundary btw! You can split your system into subsystems (microwhatever ;-)) and then define contracts among them using clojure.spec. BTW I can definitely see where your comment is coming from: I’ve played around with strongly-typed PLs in the past and just a couple of days ago watching https://www.youtube.com/watch?v=RFrKffrKCeU I considered (once more) start playing with another language
A funny thing happens when you get into Clojure. Something about the language (immutability, macros, REPL, ?) promotes higher-level thought, better solutions, and less code. Spec (at-the-edge and where-interesting, but not everywhere) and clojure.test are appropriate, but you also have this weird phenomenon where you get further before the headwinds of size kick in.
P.S. At a Conj, someone asked Stuart Halloway (who wrote the 1st edition of Programming Clojure) something skeptical along the lines of, how did you ever reach the point of not making type errors anymore, to survive amidst dynamically-typed notation? His answer: You make type mistakes! So what? You're at the REPL, and you're working bottom-up, form by form, and you run the form you just punched in, and you make it work, and then you move on! The instant iteration is better in every way than larding the code with types.
I like Stu's answer there -- that fits with my intuition (mentioned above) about working in the REPL with small functions, making them work, and then either not changing them again (because they're small and simple and do exactly what they need to) or using the REPL to explore what those are and, again, making them work and moving on.
I very rarely run into type errors these days. I still hit NPEs occasionally -- and always when dealing with either numbers or strings because those dip into Java interop and a pure Java implementation that doesn't like nulls. My Clojure code is full of nil-punning tho' and that's idiomatic: nil
is your friend!
@U06C3GCND3J If you want to pair for 10 minutes and compare how I do something in cljs vs how you would do it in type script i think it might be educational for both of us. I think it's more akin to catching a ball then people want to admit, you can chat about it all day long and you still wont be any better until you get on the field and get your reps in. I think you will be surprised, watching me try to program in a typescript, just how much of the type system relies on you knowing how it works. It's the same way in cljs, you have to meet it midway.
Sure! I’ll shoot ya a DM and we can pencil in a time
Hi all, what is the JVM system property that forces Clojure to keep all local variable values up the call stack from being cleared, to aid with breakpoint-based debugging? I know it exists (have used it in past jobs/projects), but right now, my Google-fu and ChatCPT are both failing me. I feel like this had the word “locals” in it…
I read somewhere about putting error data in the return value rather than throwing an exception. Is this common practice and pragmatic in Clojure?
It's not the only way to do it, but it can be pragmatic. As one example clj-http let's you choose between errors as data or errors as exceptions, https://github.com/dakrone/clj-http?tab=readme-ov-file#exceptions.
@U7RJTCH6J Thanks for the link! Those are good examples of the different approaches. Do you think it's easier to compose fns that return errors instead of throwing? Is it purely a matter of preference or is there more to be said on this?
Is a fork better than a spoon? It depends on whether you're trying to eat soup or salad. One verrrry coarse way to think about it is whether or not the error can be handled locally. If the error can be handled locally, it's often easier to have errors as data. If the error can't be handled locally, then exception might be easier. Maybe.
Error handling has come up a number of times in the slack I get the feeling folks want to learn the easiest or best approach to error handling. The problem is that there is no easiest or best approach. Almost by definition, error handling is a context dependent, stateful, cross-cutting concern. It's very messy. It also tends to be severely under-documented. It's not uncommon to have a specification for exactly what to expect when things go well, but it's extremely rare to have a full specification for all the different ways an operation can fail. It might be a useful exercise to read the man pages for various POSIX file and networking functions (eg. man 2 write
).
The canonical work on error handling is http://erlang.org/download/armstrong_thesis_2003.pdf which isn't really about errors as data vs errors as exception, but about how to build reliable systems in the presence of failures.
At least in clojure, I've started using ex-info
a lot more when throwing exceptions so that I can include relevant data.
I've liked https://github.com/scgilardi/slingshot when I've used it, but I mostly write libraries and it's an extra dependency that I don't necessarily want to push onto others.
At my job, we’re not consistent about it but we generally try to keep to a couple rules: broken invariants should throw, errors that are caught and dealt with in a single function shouldn’t throw (just track the error with nil or a qualified keyword), and throw if crossing domain boundaries (a route handler calls into a service, the service should be opaque so don’t put service-specific logic in the route)
More generally, when there’s no ADTs or pattern matching, error handling without exceptions is just more cumbersome. You need to either have a bespoke error type/shape or rely on nil-punning to stand in for errors, and both can have higher costs than throwing an ex-info with good data attached.
Thanks very much for the thoughtful replies, @UEENNMX0T and @U7RJTCH6J.
I had looked at Slingshot before, but settled for https://github.com/exoscale/ex because it seems to be more actively maintained and the semantics appear more familiar to me. I do wonder why they suggest not to create your own error types (descendants):
> The type key should either be one of our base type or a descendent. Descendents should be avoided when possible.
My thought was to catch all exceptions that arise when communicating external systems and to define layers of descendants like :github-api-error
-> :api-wrapper-error
, which would allow me to get a big picture view by looking at the high-level errors but also to increase resolution and look at specific error details that are lower in the error hierarchy.
Your explanation of the challenges in error handling is very good, @U7RJTCH6J. I think the part of "no easiest or best approach" is what I'm finding somewhat fatiguing in my implementation. There are so many potential approaches and I've started off with some inconsistencies and consolidating it all to a single model so I know what to expect has been kind of hard. I guess it's about making a decision on a model and sticking to it.
oh ex
is cool, i've not see that before. we use slingshot at my job but it's kind of unwieldy. that looks much nicer.
@UEENNMX0T some others I came across that might be of interest to you:
• https://github.com/fmnoise/flow
• https://github.com/IGJoshua/farolero
For me the main drawback of these two options was that they're kinda designed around ->>
but lack ergonomics around transducers. At least that was my impression when I looked at them before a little while ago.
I know and like farolero quite a bit, but both it and flow are big changes in how they work compared to switching from "slingshot" to "ex"
It's a little easier to offer advice for problems in a specific context. We can try to see if there's any more specific advice if you're able and interested in describing your particular use case a bit more.
I recently stumbled onto https://github.com/BrunoBonacci/safely. I haven't tried it, but it seems neat and I might give it a shot on the next project that seems relevant.