This is typically the case where Clojure really shines, so I’m surprised you didn’t find more help in Clojure’s tooling. Are you using the REPL at all?

Your case sounds like a dream scenario for Clojure to me. I would say that working with the functional core of a rich business domain, using pure functions and data, is right where I would want to be. It is also usually very fast. Maybe your typed clojure and pre-condition predicates are slowing you down more than helping?

@U9MKYDN4Q Yes, I am using the REPL to run the tests for the current namespace. Do you have experience with this domain or is it a theoretical "case where it shines"?

@U07FCNURX The pre-conditions and spec definitely speed up things a lot since the help pinpoint the location of the no-longer-valid assumptions in the code when it is refactored. Without these, the nil-punning would just forward erroneous states through lots more code before something finally breaks.

Case where it shines: domain-code heavy. At least in my understanding of that concept, which would be mostly pure data transformations and decision making code.

By “use the REPL” I’m thinking of more than just tests. Evaluating smaller parts of your code, and interacting with the process as you build it.

Well, I use the REPL to run the tests - you can think of the tests like a persistent repeatable REPL if you like 🙂

I can think of the tests as a set of pre-conceived examples of using the code. The REPL allows you to easily try more stuff from different angles.

I have a lot of experience with both tests and interactive REPL-development, so I have a pretty good understanding of what each excels at, at least for my experience.

One example, I use generative (property-based) testing to make sure all cases are covered in e.g. multi-methods. But this is just a work-around for the lack of algebraic data types and the much faster feedback you would get from a compile for e.g. incomplete pattern matching when adding more variants to some data type.

To me it seems like you've made up your mind.

It’s very hard to offer any good advice for such a broad problem description, but there is something about a 25kloc code-base with 500 line stack traces and minutes of latency in developer feedback that makes me think this code-base hasn’t been written with the Clojure philosophy as a guiding principle.

Speaking without context (since I've just got to go on what has been said in this thread), one suspicion I have is perhaps the boundaries inside the architecture are not well defined. If you don't have a strict enough separation between the interpreter, core engine, rules, etc. that evolving one means you have to go and change all the others... this will be much more painful than if your language compiler kept yelling at you that you missed an edge case in your ADT.

Since @U07FCNURX and @U9MKYDN4Q are not doing it themselves, I can do it for them:slightly_smiling_face: Have a look at their screen cast https://www.parens-of-the-dead.com which shows what I believe to be their day to day workflow. It doesn’t perhaps show so much repl driven development (I believe @U04V70XH6 has a great talk on that) but it does show how to work with Clojure and tests and stuff.

While the screencast might show some fun times with Clojure, I think the problem space is too small to answer the initial question here. But I do have experience working on a large Clojure codebase with large and complex domain logic. It's been a wonderful coding experience these last 10 years working on it, since the Clojure way of working is so well suited to the functional core/imperative shell architecture, and the pure functions and immutable data makes most problems very local in scope.

hard to give you valuable precise answer without seeing the code, but I assume the same what was already said. From what I saw software developers tend to keep too many code into 1 ns, because they manage ns wrong and are trap into ns dependency loop. Split code into ns correctly, refactor it to not repeat code, make it readable and simple. I assume it should be all what you need. Here is how I like to make ns, this is a part of my hobby application:

├── binance
│   ├── auth.clj
│   ├── historical_data.clj
│   └── market.clj
├── data_collector
│   ├── binance
│   │   ├── api.clj
│   │   ├── biz.clj
│   │   ├── db.clj
│   │   └── integrant.clj
│   ├── db
│   │   └── postgresql
│   │       ├── integrant.clj
│   │       ├── json.clj
│   │       └── utils.clj
│   ├── integrant.clj
│   └── start.clj

@U9MKYDN4Q the code is written in an idiomatic style. (Typed Clojure was a bit of an experiment to see if the application would become more maintainable but don't let that distract you, we do not use it pervasively, I am happy to elaborate on that in another thread).

@U0WL6FA77 agreed, the application is structured according to the domain. A common refactoring is breaking a now-it-has-grown-too-large namespace into smaller modules, even if the Clojure style is to have quite large namespaces, I like to keep them specific and cohesive

The result is not typical of idiomatic Clojure in my experience. As an example, 500 lines of stack traces seems to indicate a much deeper call-graph than I see in most code-bases I frequent.

@U9MKYDN4Q the deep stack traces happen mosly in the interpretation phase from e.g. laziness and higher-order functions and the nature of the domain, but like I said - is there a good way to evolve and maintain Clojure code efficiently in this type of applications or have we grown out of its sweet spot?

The same question was a topic for an evening in our local Clojure user group and the discussion was lively but without any definite good answers but people agreeing that they run into scaling pains.

I don’t agree with the premise that Clojure has a sweet spot that you grow out of with enough code or business logic. Other than that it is very hard to give any specific pointers based on such a lofty description.

And my gut feeling is still that if your call stacks keep growing deep as you evolve your system, then maybe things are not properly pulled apart in the design. In my experience, Clojure code bases grow in “width, not height” as they accrue features.

Well the core interpreter iterates over event streams so we cannot avoid deep stacks since there is recursion. This is not a problem per se, but I used it to illustrate that we are not the typical Ruby on Rails "to do" list application with 10 lines of code per URI endpoint.

When working on a problem in your codebase, can you keep the essentials in your head? Or do you feel like you're teleporting between disjoint systems and constantly swapping something in for something else?

@U05476190 it fits in the head.

Can you tree your project to show files structure? It is very theoretical discussion without any strict data input for this discussion. I mean it is impossible to discuss at some point without seeing the project.

Example of something slow: a refactoring to change something from a number to a map, e.g. let's say I want to have a money with amount and currency instead of just numbers. This touches a lot of code, but how do you know where? So, you say, let's start by adding a money function and data structure, and a predicate. Then introduce it in the ledger for the financial reporting. The code still runs but does not give the right results. So you add predicates to check that nothing goes into the financial reports that is a number but only the new money data structure... No run the tests. The compiler is not helping but at least the predicate will help you find the call-sites that provide data for that. Now move up the stack, fix things. Rerun tests. Repeat repeat repeat....

maybe because of tests? Again this is my personal preference, but I focus mainly on full request->response tests. I don’t do unit tests for everything. Only for things which need to be documented by tests. I mean if you have full tests request->response or if it is not API processing data from the beginning to the end the test will fail and you will know there is something to fix.

We have tests at different levels, e.g. integration test on the API endpoints to make sure everything fits together and module-level tests to make sure the key bits fit together, e.g. the domain is commercial contracts, so for each contract implemented in the DSL there is a test suite that interprets the contract against event streams and validates that we get the expected outputs. So the "unit" is a module, there is less testing of individual functions except the critical ones since the too-narrow-test to code would create additional coupling that would make refactoring harder. Also, we use generative tests to see that the code does not break in scenarios we did not consider and that e.g. all multimethods are implemented for all relevant variants.

So definitely, automated testing is one of the enablers for scaling Clojure to this type of problem. We could not do without that.

Well if you are really doing all things which were mentioned in good quality and it is still complex, than in other language it could be even worst. Although think about this later if it is really done like that. I am sorry, but I don’t think I can say anything better, than I already said without seeing the project which I of course understand is not possible. I don’t think I can help more.

> Example of something slow: a refactoring to change something from a number to a map I think specifically this kind of refactor is always going to be slower in a dynamic language comparing to a strictly typed one, because you are changing types. In a strict type lang you spend a lot of time specifying what should be what type and the compiler was built around that, so will always be easier IMHO. One thing in my experience that helps with this situation is to build around protocols, but I know is hard to predict where you are going to need it (like in the money example you just gave)

but again, is a specific type of refactor, which is changing a number for a map. With numbers you never tend to program against protocols, so they are harder to change

Another thought about these kinds of refactorings. I see this when I attend conferences where there is a majority of statically typed presentations, and what I hear from stage is “So I change this type here, and I see all these thousands of red squigglies all over my code, type safety FTW”. My thought seems always to be but why does a change like this affect the code in so many places throughout the code? Could the code have been organized differently so the impact of such a refactoring (yes I understand it’s but an example) would be more local? Like, for your example, only visible in the currency ns?

And, here I feel there is a tension between what’s easy and convenient and what’s too many layers of indirection. So if you start to implement currency as a number, then of course (+ amount1 amount2) makes perfect sense, and introducing an indirection of say, (currency/add-amount amount1 amount2) can be seen as superfluous future proofing. But, perhaps as time goes by, this is the exact kind of indirection you need. Good design is about choosing the correct indirections.

@U04V5VAUN True. In fact we moved money from numbers to maps and functions like money+ money-max? etc. It breaks the primitive obsession anti-pattern. Also, many (not all) incorrect uses of functions are reported by the compiler, so it gives a much faster feedback loop.

But the "right" abstractions are something you discover over time, and they are not stationary since your understanding of the market, regulatory requirements, customer demand and requests etc. also evolve. So to scale it, the code base must be malleable and for this we need a bag of tricks to make it so. Hence the question.

I guess most of those problems arise when dealing with primitives (nums, strings, etc) because for the rest we are already working against abstractions, changing a map/vector/set to something else is a matter of implementing the right interfaces on the new thing

The slow feedback loop definitely sounds like a problem of the code base size and domain complexity. Using a typed language isn’t going to solve that. It might help with certain type of re-factors, but optimizing for those means you’re loosing out on interactivity which can speed up the both refactors and new feature development. You said that you use the REPL to run the ns tests. In my experience that’s usually too big of a scope. I try to run the ns tests once to see all the things that are broken. I then dump the results into a scratch file and start running the tests one by one while working to fix each one. Then when I get through all those, I run the ns tests one last time as a sanity check. This tend to speed up the feedback loop a lot. Hope that helps a little.

Or if even the individual tests take a while, like generative tests sometimes do, I just make a quick example invocation that I run over and over while fixing the issue.

Based on what you're describing, it seems like you're hitting some combination of the <https://en.wikipedia.org/wiki/Inner-platform_effect|inner-platform effect> and Morris's corollary to <https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule|Greenspun's tenth rule>: &gt; _Any sufficiently complicated <https://en.wikipedia.org/wiki/C_(programming_language)|C> or <https://en.wikipedia.org/wiki/Fortran|Fortran> program contains an <https://en.wikipedia.org/wiki/Ad_hoc|ad hoc>, informally-specified, <https://en.wikipedia.org/wiki/Software_bug|bug>-ridden, slow implementation of half of <https://en.wikipedia.org/wiki/Common_Lisp|Common Lisp>. (including Common Lisp)_ Hard to say for sure without knowing more about the code, but it's possible you're hitting the limits of Clojure because you've made something so malleable that it has had to take on features of a programming language. If your use cases for that language are primarily driven by business use cases, it could be developing in a piecemeal fashion without proper consideration of the PL and language design aspects. No programming language, even ones designed from the ground up for authoring DSLs like Racket, will necessarily prevent this type of complexity creep from overwhelming your development process. As others in the thread have suggested, it is not yet obvious that a domain-specific language and interpreter is actually required to model the business logic you're describing, but without seeing any code it's impossible to say anything definitive.

@UFTRLDZEW It is always a danger that a DSL tries to become a general-purpose language. We avoid that by using the DSL as the skeleton and implementing custom logic as plain Clojure functions. Then the DSL language can focus on things like, obligations (deliver X by date D), a discretions (customer may buy A for price B with a penalty of P if not delivered within 10 business days) that are truly general and avoid the inner-platform effect you describe.

The interpreter is necessary to evaluate the contract against the facts of what happened to tell the parties, "the invoice should for this month should be like that", the KPIs came out to this, with these financial implications, "this thing happened three months in a row so you now have the right to ...."

I've never used a rules engine, but what you're describing sounds like the type of problem that forward-chaining was designed to solve – figuring out all the implications of a new piece of information in a declarative way. Have you used one of the rules engine libraries in Clojure? Were they insufficient in some way that made a DSL necessary? https://github.com/cerner/clara-rules

https://frankiesardo.github.io/minikusari/#!/minikusari.intro May also of interest for the Datalog integration.

We use core.logic for one of the mini-languages in the front-end to show various supply-chain relationships. We did evaluate Clara rules and Riemann for event processing, but it was not a clear fit for the specific use case. These are interesting topics in their own right, but a bit tangential to the topic: how do we make Clojure applications scale without incurring too much friction and effort as the application evolves and grows?

I don’t think they are tangential; the question of how to scale an application as it changes depends on the application, its use cases, and the external systems it must interact with. I’m not sure it can meaningfully be answered in the abstract.

Certainly there are approaches and tools that try to make modularity easier (#polylith comes to mind), but even the term “scaling” itself has multiple meanings depending on the context, which all pose rather different challenges: • “scaling” the application to deal with greater volume, velocity, and variety of requests as they grow • “scaling” the application to take on additional business use cases or contexts • “scaling” the application to integrate with new systems or sources of information One rarely encounters any of these in total isolation, but IME they all suggest pretty different strategies to deal with the added complexity they bring, making a general answer difficult.

https://polylith.gitbook.io/poly

You seem to call out the only example case that static types would make easier: "I want to change the type of something that flows through my functions" Yes, if you cherry pick the only use case that a static type checker solves you will easily convince yourself that Clojure cannot scale. For every other problems, Clojure will be much better at. Also, don't fall in the trap of scape-goating Clojure, which I think you are falling into. Just acknowledge you, for no technically justifiable reason, would maybe prefer working with a typed language, simply because you like it better! That's ok, don't try to spin it as a deficiency of Clojure. Because I and other Clojure devs would have no issues working on your code base, delivering business value at a high velocity, and having low defects.

Your code base seems complex on its own, you will face challenges working with it even in Idris. It sounds like a complicated non-trivial piece of code and logic that does a lot. It can be tempting to assume the reason some things are difficult is because "Clojure". Again, of course you would also struggle in Java, Idris, Haskell, F#, etc. But you just wouldn't have an easy excuse in thinking it's because of the lack of static types. Instead you'd just think, ok ya, this is non-trivial logic, and you'd just roll up your sleeves and do it. This is a common trap. I've seen teams rewrite from Clojure to Java, and have no impact on their velocity, just took them back months because of the rewrite. I've actually seen decrease in velocity, and sometimes higher defects. This was from real data, real team velocity measures and number of tickets cut. The most surprising is how much the kloc grew haha. But, if the team prefers it in Java, and they are more motivated, and they stop complaining, that's still a win, team moral and your moral is important. Just don't blame the language, that's what I have issue with 😛

Ok, with that out of the way haha. I do agree with others, the deep stack sounds like a possible problem. Recursive functions shouldn't matter here, because refactoring that is just one signature you change. But if you're saying you need to change a number to a map, why is that flowing through so many things? In Clojure, functions are not coupled to ADTs (abstract data types) like they'd be in Haskell or Java. Effectively, each function gets its own input structure. So if a function now wants a different input, the scope of the breakage should be restricted only to itself and its direct callers. That function can continue to call the functions it depends on with the input they always took, and the callers can continue to be called with the input they always took. If that's not the case, it means that you are taking that input structure and passing it down to other functions, that pass it down to others, and so on. Or you're injecting the same structure into multiple functions at the top. This is normally a bad practice in Clojure, unless they are actually modeled entities, aka they'd have a constructor, a spec and a consistent name across the code-base.

Alright, now something else. For a change in type, you just need to exercise the code paths once. Your tests should already cover most of your code paths. And also just try running your app and manually test a few things. That should catch most breakage from the type change. Then go fix them. That's because Clojure is strongly typed at runtime, just not at compile time.

Other then that, ya, there's not much more you can do, if you change money from number to some structure that has currency on it, and you want to update all the logic that uses money to be currency aware, you'll kind of need to be a bit more surgical in finding all those places. What's hard to see though here, is that, while this particular change is a bit more painful then if you'd had typed everything from the get go and had a static type checker, is everything else you benefited from using Clojure... Your total code size is dramatically reduced. Your data is immutable avoiding you a ton of mutation at a distance problems. The dynamism lets you easily handle things generically, and limit the explosions of functions specialized for every combinations. The REPL makes it a lot easier for someone to learn the code-base, and understand the actual business logic, make sure it works, etc. You probably didn't have to refactor as often either, because Clojure is more resilient to breakage free accretion. You probably benefited a ton from macros and lispiness in being able to build this DSL in the first place. And all that.

I say this, because I think the things Clojure helps with, are amortized over all your code changes. So in a way, they are subtle, because it's not like: Hey you know this ONE thing that's always so annoying? Clojure gets rid of it! Instead it's more like, all your code changes will benefit from the general simplicity of Clojure and the flexibility and power it provides. And static types falls in the former. It goes, hey you know that one time you need to refactor money from an int to a currency aware structure? And how it's kind of annoying finding all the places that used it to change them? Well Haskell solves that!! WOOT. On the flip side, the downside of doing all that typing and the restrictions it provides is amortized over all changes. So the negative is more subtle, and the positive is very in your face. So it's more obvious. But if you think about it in terms of metrics, like realistically how often you have that kind of task per year? And really how much longer did it take you do it than if you'd had static type? At worse a 1 week task takes you 2 weeks. At best it took you kind of the same amount of time, but just felt a bit more tedious in how you went about it.

P.S.: Sorry for the strong response haha, it's just this "maybe Clojure can't scale to large projects" is a recurring theme, you were unfortunate to be the one I address it too. So sorry you were the one that has to take in my wall of text about it 😛

I'll just note -- since I got @-ed into this thread -- if you're changing something like money from a number to a hash map, you need to change the math expressions that operate on it but you don't need to change any of the code that just passes that data through whereas in a statically typed language you would need to change everything in the call tree, not just the math. On code size, I'll note that we have over 130K lines of code, organized using Polylith, from which we build nearly two dozen "apps", from over 100 "components" (small subprojects). And we maintain all that with two developers. So, yeah, Clojure scales just fine.

Practical example that has helped me with changes like changing a pervasive field's type: clojure.spec.alpha/fdef. I have it instrument all functions in my dev env, so args are checked every time they're called. But it's easy to overdo it with specs. The "functional core/imperative shell" paradigm helps here, by indicating where your specs will have the biggest impact: on the interfaces between the functional core and imperative shell.

> "Well, I use the REPL to run the tests - you can think of the tests like a persistent repeatable REPL if you like" That's true, but not REPL driven development. I also run my tests from the REPL, but it seems really odd for that to be the primary use of the REPL. It's much faster to iterate, with sub-second feedback, while building up a form, layer by layer, before it actually goes into a function or test. Unit tests are only the second layer of testing, after REPL, which I'll (maybe) run before committing a change, and then only the unit tests for the current ns. End-to-end and integration tests can happen later and asynchronously to my dev "thread". REPL feedback that takes more than a second or two is a smell, IMHO.

@U04V70XH6 I will take a look at Polylith. With your experience, what are the key things you would ask a new team member to put into the design that makes it easily refactorable?

@U0K064KQV thanks for the energetic response. If we just focus on Clojure, with your experience, what are the top tips to keep a Clojure code base in a good state where you can refactor it easily over time.

@U040NGRHA4B "refactor" is a broad space -- keeping domain functionality grouped together helps: for your example, if all your money operations were in one namespace that everything else called, you would only have to update that one namespace to change from a number to a hash map.

@UJY23QLS1 I did try the spec fdef but in the end I found it too clumsy, so now I am mostly using spec and generative testing, but not that last bit is not something I write any more.

(at work we have a ws.currency ns containing all our currency-related stuff, for example)

Just from looking at my code, here are some common refactorings that I do again an again: 1. change something from a primitive to a composite (e.g. replace number with a money map)

(we actually have ws.currency and ws.money -- the latter uses Joda Money types via interop and no code anywhere else performs operations on Joda Money so we could easily switch that to a completely different representation without touching any other code)

2. "ADT/pattern matching" extend some concept with a new variant and make sure that this new variant is properly implemented everywhere it is relevant. An example could be adding a data field type to the ontology and being sure that the documentation is properly generated, Information Schemas are built, API support is in place, the mapping to and from external data sources of said field type, example validation and generation for documenting it etc.

3. collect multiple fields into one concept, e.g. may maps have let's say a source and a reference, perhaps we should collect that into a contract-reference concept (this is a variant of 1, switching from primitive to map, here we are taking multiple fields and rolling into their own concept)

If you're finding that such changes affect "lots of code" then you probably aren't grouping things per domain concepts...

The first is very common, it can be money, but also "customer" could be a string, but perhaps better an entity. Now, that Description is a string, how do we represent that in multiple languages, .... this is almost a daily task...

@U04V70XH6 as for domain concepts, one thing that I find very useful is to have a predicate for every domain concept. Even if you use e.g. a number for money, write a (defn money? "is x a monetary value?" [x] (number? x)) so you have a minimal definition of the semantics rather than using just primitives. Same for the collection types. Then use these where you depend on the that concept, (fn discount[list-price] {:pre [(money? list-price)]} ...) This provides a basis for growing it into a full-fledged domain concept later, including switching from primitive to structure data types.

Yeah, predicates -- or specs -- can be used for that -- for functions that actually care about the values passed in (and not just pass through to other functions). Predicates are substantially more powerful than types -- but you need to be careful that your predicates are cheap to execute if you have performance-sensitive code.

> '2. "ADT/pattern matching" extend some concept with a new variant...' Multi-methods?

Yes, it feels wasteful to use these spec-based predicates so much, but like my old colleage said, you cannot put CPU cycles in the bank

@UJY23QLS1 > Multi-methods?

Multi-methods are fundamentally different from ADTs since they are open, not closed. ADT have the benefit that you can define them to say I care about these seven variants and if I add another I want to know where it is used so I can decide how to handle the eight variant when I introduce this.

So what I do, is I test the multi-methods via generative testing to make sure that I test all variants up to an including variants that I add later.

The feedback cycle for this is not as fast as compiler errors in languages with ADTs, so it is a point of friction

@U0K064KQV > For a change in type, you just need to exercise the code paths once. > Your tests should already cover most of your code paths. And also just > try running your app and manually test a few things. > > That should catch most breakage from the type change. Then go fix them. Yes, this works. But it is a slow feedback loop - run tests, see one error, dig through a stack trace if you are lucky enough to break an assertion, or try to reverse engineer what went wrong from some weird generative example or work backwards from a test saying, I expected 400 bananas in this account, now there are none and trying to figure out where the error is based on that. My quest is to find out how to write code where this is not such a chore

How high level are the generative tests? Having to dig through a stack trace from a generative test sounds like a terrible experience. Usually generative tests should go at the same level as unit tests and cover fairly low level functions.

@U2UKX5DQR agreed. To make it focused there are gen tests for e.g. an invariant like, "we can generate an example value for every field type allowed in the ontology" (testing a single multi-method), to "every valid ontology can be exported to an Excel sheet" which is an important invariant but is much higher level exercising much more code. The latter will give a "minimal" example of something that breaks the invariant, then it is often possible to reduce it even more manually, and then pinpoint and fix the issue.

@UFTRLDZEW > even the term “scaling” itself has multiple meanings depending on the > context, which all pose rather different challenges: > • “scaling” the application to deal with greater volume, velocity, and variety of requests as they grow > • “scaling” the application to take on additional business use cases or contexts > • “scaling” the application to integrate with new systems or sources of information > I am asking about the middle one - adapting and extending the application as we uncover the requirements from the market. I am looking for ways to make the code more refactorable without too much friction and papercuts with the Clojure toolstack.

1. change something from a primitive to a composite (e.g. replace number with a money map) 2. "ADT/pattern matching" extend some concept with a new variant and make sure that this new variant is properly implemented everywhere it is relevant. An example could be adding a data field type to the ontology and being sure that the documentation is properly generated, Information Schemas are built, API support is in place, the mapping to and from external data sources of said field type, example validation and generation for documenting it etc. 3. collect multiple fields into one concept, e.g. may maps have let's say a source and a reference, perhaps we should collect that into a contract-reference concept (this is a variant of 1, switching from primitive to map, here we are taking multiple fields and rolling into their own concept) For #1 and #3, to be honest, I'm surprised you have these come up so often. That's my first surprise. It makes me wonder, what do I do in my design and code structure that I just don't have these come up often, but I don't know, I just can't say I face needing to do these very often. For #2, that sounds maybe specific to your code base and the DSL? I'm guessing if you add a new DSL keyword or something, you'd like to make sure it's handled appropriately? To be clear, all types in Clojure are ADTs, as they are the Union of all other types, any variable can be a sum of all types. What you can do if you wanted to case switch in an exhaustive way and have compile time detection for this is use a macro such as: https://github.com/lambdaisland/uniontypes or write something similar. In general though, I've preferred making these open and having default handling. Since a lot of time you add an option but it only really matters in specific places, and being forced to handle it everywhere when the default would work can be annoying. But its very use-case dependent. Alternatively, you can use polymorphism instead, through multi-method or protocols (this is probably the more idiomatic way). You then do something like:

(s/check-asserts true)
(s/def :some/case #{:a :b})
(def foo nil)
(defmulti foo #(do (s/assert :some/case %) %))
(defmethod foo :a
  ([_] (println "A")))

#object[clojure.lang.MultiFn 0x4a45ff31 "clojure.lang.MultiFn@4a45ff31"]

(foo :a)
A
nil

(foo :b)
java.lang.IllegalArgumentException: No method in multimethod 'foo' for dispatch value: :b

(foo :c)
clojure.lang.ExceptionInfo: Spec assertion failed
:c - failed: #{:b :a}

If all the multi-method implementations are able to stay in one namespace I prefer a function with a case. It is easier to navigate from call-sites, and as a bonus (in this case) it throws on unimplemented cases.

I would second the point that #1 and #3 above do not occur often in my code-bases. It happens, but I read previously that this was “almost daily”, and I’m nowhere near that.

For the closed set polymorphism (the number 2 "ADT"/pattern matching) both using case or adding an assertion to the dispatch function in a multimethod are solutions, but both have the "problem" in terms of efficiency that they defer the problem to runtime. Now then, to see that you are missing implementing the new variant various places while writing code you need all tests to cover all cases including the new one. So now you have to use generative testing. Then you will get a stack trace with the error, and then you can fix it.... So yes, you can make it work but it does not feel very efficient as a developer ...

@U0K064KQV The reasons these refactoring come up so often is that it is an early stage product that we need to bring to market, it means that we are uncovering the requirements and reshaping the requirements as we go.

However, my experience with Clojure over the years in projects from a single person to hundreds is that people don't do a lot of these refactorings, they stay with the primitives rather than introduce proper domain concepts (this primitive obsessions is also a thing in other languages, enough that it is a named pattern). I see this as a signal that there is too much friction to refactor to a stronger domain model (this problem is not exclusive to Clojure, but the tooling in other languages removes some of the friction).

@U0K064KQV > Now with the defmulti, what I normally do is I have the multi-method and > all implementations together in one namespace. So it's kind of obvious, > add a new case, add a new defmethod. But even if you prefered to spread > them out, all you have to do is a project search for: (defmethod foo > and you'll find them all and see if it's missing a case. To me, it's > just a non-issue really. You're the one who added a case to it, so like, > why did you add it if you don't plan to implement a method for it? It is a slightly different problem I am talking about. I am introducing a new say type of fields to the ontology (e.g. you can now declare fields that are enums of string values). Multiple multimethods may dispatch on field-type and field-type now has a new variant. How do I find all the e.g. multimethods that need to be updated? Yes, I remember I added the enums to be able to express something, and that perhaps the associated documentation generator, but what about that export your information model to a spreadsheet function? Or the mapping to and from external file format for the I/O of the data described by the ontology? So basically without generative testing I would have to know or search a lot.

Some food for thought: would you say it is accurate that "fields of the ontology" are a core domain concept? I could argue that modeling complex fields via many multimethods peppered across your codebase could be also considered a case of "primitive obsession". Perhaps if these fields are changing so often, they should be modeled as first class domain concepts, and things like rules and reports would access them via indirection (ie. some kind of centralized repository/namespace that proxies between the various system use-cases and the actual definitions - which can in fact be moved to code, config, or even a database). Then it would be easier to define a concrete set of protocols and contracts that all fields must adhere to and the rest of the system can "discover" new types easier. You could even flip the responsibility: enforce that any new type added must define all the following contracts/protocols/etc. in one place to be considered as a good citizen of the system (instead of enforcing that the system adapt and works with the new type added). > However, my experience with Clojure over the years in projects from a single person to hundreds is that people don't do a lot of these refactorings, they stay with the primitives rather than introduce proper domain concepts My experience has been slightly different, motivated often by the idea that if the core business is talking about X via nouns, there probably exists a group of namespaces named after X that should proxy how the rest of the system discovers and interacts with X. I think @U04V70XH6's comments about ws.currency and ws.money reflect a similar world view.

@U05476190 The "ontology" here is more the case of a meta-model that is used to describes the facts that are relevant to specific contracts. It is not a case of primitive obsession. The cause for change is a normal process of evolutionary development like, "oh, if we had a meta-model concept of enums we could provide better parsing and validation of external data than if we had only unconstrained strings".

I think you are describing in a sense the Visitor pattern with protocols as a way to do constrained polymorphism and pattern matching, e.g. "ADT/pattern matching". This is possible too.

@U05476190 > My experience has been slightly different, motivated often by the idea that if the core business is talking about X via nouns, there probably exists a group of namespaces named after X that should proxy how the rest of the system discovers and interacts with X. I agree that it should be like this with well defined, localized abstractions. However, is just that applications start on the simple side and then grow into a more and more robust domain model (or more likely, ossify). My core question is simply how to effectively move the application code into a better state as we learn about the problem and solution space and discover better abstractions?

Incidentally the visitor pattern is disliked by the OO people for not adhering to the open closed principle (open for extension but closed for modification), but in my experience precisely the constrained polymorphism ("enum" with pattern matching) where we want to ensure we consider all relevant variants is much more common in practise. This is why I like that in e.g. ML languages the code breaks if you add a new variant and dont have a corresponding match clause.

There is a thing with protocols though:

(defprotocol Hackable
  (foo [_])
  (bar [_]))

(defrecord BlahBlah []
  Hackable
  (foo [_] "foo")
  ;; notice how bar is not implemented
  )

(->BlahBlah)
;; => {}


;; This fails
(bar (->BlahBlah))

; Execution error (AbstractMethodError) at ....
; Receiver class ...BlahBlah does not define or inherit an implementation of the resolved method 'abstract java.lang.Object bar()' ....

From following this discussion, one thought I had is that it seems like you are reacting a little late. I mean, sure, a compiler-enforced pattern matcher would better help you out in your current predicament, but instead of just looking at technical solutions to your current pains, maybe also analyse how you came to be here in the first place. Seconds long feedback loops and almost daily wide-reaching refactorings have never been the norms of any Clojure project I worked on, small nor big. And this is the reason why I ask if you are reacting late - possibly you have missed out on making some bigger improvements along the way to avoid ending up in this place. I know this isn’t helpful now, but my point is that it is futile to blame your tooling for problems with your process. > I agree that it should be like this with well defined, localized abstractions. However, is just that applications start on the simple side and then grow into a more and more robust domain model (or more likely, ossify). My core question is simply how to effectively move the application code into a better state as we learn about the problem and solution space and discover better abstractions? I guess what I’m getting at is: make design improvements in time 😅 As a dynamic language, Clojure will never be your best option for fearlessly restructuring 25kloc of code with little or no isolation between components/sub-systems. I doubt you will find a tooling answer to this question, so obsessing over programming language primitives won’t help you much. I think @U05476190 advice is sound, and probably your best bet at this point is to see if you can identify some bigger improvements in the overall design that can improve isolation and help you reduce the frequency of refactorings that touch a lot of code.

@U9MKYDN4Q I think we agree. Knowing the code, I am making design improvements in time but I am also paying the price so that is why I am asking, given the constraints of Clojure the language and toolstack, is there a better way to do it? Are the some practises that make this faster or more easy?

As for "wide-reaching" refactorings, since I know the code, I would say more precisely refactorings where the impact is not immediately clear if there is no generative testing in place to exercise the most relevant code paths.

Like I said, I don’t think there’s a technical answer to this. I’ve never had the kinds of problems you describe, so my suspicion is that you’ve made other choices with the design than I would have. It’s hard to tell for sure without knowing the specifics.

What type of applications are you normally working on?

business systems, web applications, developer tooling, games

So what do you tell a new developer joining your team? Hey, here are a few golden rules so we can keep this maintainable as it grows and the requirements change...

Things that have already been said in this thread 😅 Try to encapsulate concepts in local islands, be wary of concepts that spread across the entire code-base - especially “magic” primitives. Use multi-methods and protocols sparingly, and primarily as an extension/integration mechanism. Write as much of the code as possibly as regular pure functions. Cater to the developer ergonomics (e.g. it’s easier to navigate to a function than to a specific multi-method implementation). Model as much as possible with serializable data. Avoid deeply nesting abstractions - model relationships as flatly as possible (I do that with data as well as namespaces, code, and abstractions).

Re: the last point, it’s better to have one “orchestrator” type function that calls five others and piece together the results than it is to have five functions that call each-other in succession.

This last piece of advice is a nugget! And following it will have profound effects on your codebase. I’d go as far as to say it sums up idiomatic Clojure.

@U040NGRHA4B I think you overlooked my link to the union types library? It does compile time exhaustive check which is what you're asking for, and you can also implement a similar macro quite easily.

@U0K064KQV Yes, thanks, it looks nice. In fact, something like that should be in core since this is such a common pattern in software development.

@U9MKYDN4Q just to have a point of reference, how many man-years typically go into the applications you mentioned for building the initial version and how many full-time employees are are assigned for the daily work and evolution of these over the years?

Your points are fine, they sum up to "keep it nice and clean" and I think I am looking at it from a slightly different angle, how do we keep it nice and clean as it evolves, refactoring being the process to do just that (nice and clean one day may be under-engineered the next day, or over-engineered if something is no longer important)

I don’t have hard numbers for you. We’re currently a team of 7 people on a project that’s been going for about 8 years

I think at this point we’re going in circles. I told you what’s worked for me in several projects.

OK, that's a good ball park for a normal business team. Perhaps we just have different context, e.g. rate of change of requirements

Our requirements change at a representative rate I’d say.

I think for me, the problems you mention are kind of small. Like again, ok you change the type from number to map. It's not a big deal, you just spend a little bit more time on the refactor than if you'd have static type. But we're talking like 1 to 30 days best to worse case scenario. I just don't find this to be a big problem. It's low impact. On the other hand, the design of the systems together, the data model, the way we're modeling the domain space, those are way more impactful. And having a simple language without a lot of cruft between me and my domain, that doesn't impose constraints on my data model, such as forcing homogeneous collections, forcing static constructs, forcing object identities, but instead where I can work directly on data and have it map perfectly back to the domain as it is, without any impedance mismatch between internal representation, data store representation, business domain representation and external representation. That's a force multiplier, the short and long term impact towards success will be so much higher here. Similarly, having a REPL driven development workflow force multiplies domain exploration and emergent design, which again, good design in how we model the domain to me is most important, so this in itself I also consider a much more impactful effect. Then having an open system, like you said, closed for modification but open for extension, that's critical to long term maintenance of an operational and live service. You're literally asking about refactors that require modifications, which goes against the open-closed principle. Accruing more features over time shouldn't require modifying the existing code, all these refractors you're doing can be a sign of a design that isn't enabling open-closed principle. You're still able to afford this level of breakage because you're in prototype phase, but once you go live, you really want to embrace a more open for extension system that limits breaking the foundation to a maximum. And then in terms of good design, that's so important, even if you're using Haskell, Java, F#, your types won't save you. If you don't follow the best practices discussed here, for example, having shallow call stacks, top level orchestration that coordinates data flow, instead of deep nested call chains, you'll be in a mess even if you've got types. In a weird way, Java lets you get away with bad design a bit longer, almost hiding the negative effects of it until it's too late. Clojure immediately makes you realize the importance of it and being careful with it, even to your juniors, you have to teach them careful and thoughtful design immediately. I guess that's my take and experience. What exactly you're doing it'll vary. But like, the kind of system you describe is very often implemented in a Lisp, Scheme, and all that, because normally you're dealing with a lot of generic types, and a lot of data-structure walking/transformations. I don't know exactly what you're doing or how it's designed, just saying a lot of compiler/interpreter are often written in a dynamic Lisp language.

Having said that, this is all very high level advice. The pudding is in the details as they say. So your specific use cases, problems, maybe they don't apply. And probably we're not able to understand the specific challenges you've got within the context of what you're doing and why. I think here the question to me isn't so much should this be in core or not, but can the language adapt to your specific needs? Or are you forced to adapt to the language? Generally, Clojure is a very flexible language, making it pretty good at adapting. In your case, if you've got good reasons, I'd say use the uniontypes library. It's a very small lib by the way, only 150 loc, like I could implement a similar macro in a day. It's not like you're taking a massive dependency. And if you find you often need to change number to map, then write your code in a more encapsulated style. Introduce getters and setters. Work with entities and add a get-number for example, so if number changes to a map the get-number can just reach into the map and get the number.

(defn make-entity [...]
  {...})

(defn get-number [entity]
  (:number entity))
 
;; Later
(defn get-number [entity]
  (get-in entity [:number :value]))

@U0K064KQV I can see you thought a lot about this. These are very deep thoughts with lots of important points. I will try to answer some of these. I think we are generally aligned in how we view the things.

Beginning from your idea to have make-entity and get-number to encapsulate the data. I really like this, and in fact I built a meta-model for this, a set of macros to define a consistently named and structured entities and value types for use in the application with the associated predicates and pre-conditions to check invariants to fail fast if the client sends incompatible data. I like to think of this as aggressive FP, since it leans heavily into using functions first, where Clojure devs often prefer maps and destructuring, so it looks a bit foreign. However, I find it quite useful since it solves my key concern: speeding up the feedback loop. It pulls the late feedback runtime errors from e.g. missing or renamed keys in a destructuring to fall into nil-punning and hopefully breaking a test all the way up to the one thing the compiler cares about, function signatures with much faster feedback. So that's a win.

A meta-model also helps with consistency and refactoring common patterns globally so it has more advantages.

For scaling Clojure development, I would definitely recommend this, manually or with a meta-model. At the bare minimum to have a "constructor" function like your make-entity and a predicate entity? to recognize that type for use in pre-conditions in all functions where the value matters.

@U0K064KQV > I just don't find this to be a big problem. It's low impact. I think Rich Hickey say something similar that he does not care about these problems because they are easy solve (even if they take time and are quite menial). So I think we agree that they are known time-sinks in Clojure development.

And I agree that in some contexts it is probably not a big deal in the aggregate. When things are stable and all the right abstractions are in place in the central bits and you are just adding extensions at the edge off of an established core you just go open-closed and extend away without too much refactoring since these modules on the edge of a system tend to be "more of the same" and with very low coupling. It is like adding a new currency if you already have the whole financial infrastructure for 10 other currencies in place.

However, and that is why I raised the question - since we are dealing with a known time-sink how do we reduce the pain it causes to the minimum, also in the short term? Are there ways to write the code that make these refactorings faster?

You rightly point to the open-closed principle for these refactorings. There are two aspects to that: First, the case of "enums" where I make a conscious choice to not design for open-closed but for a closed set. Second, for e.g. changing from primitives to data structures.

The "enum" case where it is a deliberate choice to break with open-closed is motivated by the fact that sometimes it is important to ensure that all client code that looks at some datum consider all its variants. With the macro you mentioned or pattern matching in other languages we force the next developer (or ourselves, three months later when we forgot about the details) to think about everything. The Clojure "ideology" is "why should I have to care about all these other things", which is basically the open-closed principle but the domain and risk profile may lead you sometimes decide against that in specific situations. In financial software this is often the case, if there is a variant of some input it is really useful to be able to quickly find the places in the code where you need to consider the impact to ensure you get the right numbers out.

You can still do this with open-closed (e.g. multimethods) but then you must use generative testing to ensure that all the code is exercised for all variants when a new one is added. So it just moves the problem. Then, when you run theses tests, you will get the exceptions and you can work backwards from that, This means your team has to use generative testing - something I have not seen much "in the wild", and in any case it is still a slow feedback loop. "Easy" yes, but slow.

The enum could also be built with protocols like the Visitor pattern with a macro, but again - if you have implemented the protocol for 6 variants and add number 7 to the defprotocol you will not break the code that implements it. You will not know until someone callls that 7th variant at runtime, again a slow feedback loop and it will require you to do more testing like for multi-methods.

So I think we more or less agree to the problem and the reason for searching for the better ways to deal with it.

Second point, switching from e.g. a value to a datastructure. Does it break OCP? Yes, I am breaking a contract because I want to update an internal API and no, I don't want to be backwards compatible inside my own code base, I do it because i think "tell me the amount of money and currency, don't just give me a number" is a better abstraction if someone in another country is interested. It is fairly common until you have built a stable big general product across many markets and verticals, so it is relavant to startups to be able to easily adopt now and not only get the benefit in the long run. Again, I am just making the case that I think it is valuable to look for good solutions to what I consider common problems, because I don't want to take more pain than necessary now. Yes, testing extensively will solve it, but again slow feedback loops and low-value-added work is just a papercut that would be nice to avoid.

Now the reason to get into that situation is valid. If I built a multi-currency application for a single-currency market with a single customer doing local business only it might just as well be considered over-engineering, so in the early years of a startup this kind of change is to be expected.

There is a deeper question here about designing types that we can get back to, but I noticed that Rich Hickey sometimes talks about RDF and if you take that seriously as a design principle you will go into an interesting rabbit hole of data in maps only with namespaced keys to denote the semantics of each value and strict contracts so that if the semantics of a value changes, it must have a new key. {::amount 42, ::money {::money-amount 42 ::money-currency :EUR}}so you would have OCP and backwards compatibility. It haven't really seen it adopted and it does not fit well with how we normally thing multimethod dispatch but the implications of designing like this is quite interesting.

@U0K064KQV said > If you don't follow the best practices discussed here, for example, having shallow call stacks, top level orchestration that coordinates data flow, instead of deep nested call chains, you'll be in a mess even if you've got types. This made me realise that I probably did not clarify why I mentioned that we have deep call stacks and this lead to some confusion. In fact, I agree to the design principles mentioned - it is nice an clean code. I did not call out the "deep" call stacks to indicate that the code is a tangled mess.The reason I mentioned it was to contrast it from, e.g. a Ruby-on-Rails type application where you take a HTTP request, hit a controller and run 10-50 lines of code with perhaps a few conditionals and a database query and serialize back a response. I consider these systems a "solved problem" for dynamic languages. They super useful and mostly simple with low coupling and with shallow code and the database acting as a surrogate type system.

I should probably have said that we have a much more complex domain than these kinds of applications.

More complex domain really should not lead to a significantly deeper call stack

I think the depth of the call stack takes the focus away from the real issue. If we agree that it is valid to e.g. have a number to represent money until someone in the next country asks to use the application then we also agree that we have to refactor at that point. And then the real question is - what can we do to make the feedback loops as fast as possible when we do that refactoring?

I took our app from single language to i18n, and then from one app to multiple deployments. Doing these kinds of refactorings are tedious as they touch the entire code base, but they are also few and far between in my experience.

And the call stack issue is important I think, because it says a lot about the layering. If this is the norm, than any wide reaching change will hurt even more.

Yes, so we agree that they are tedious.

Of course. I think a much more interesting question than "how do I make them less tedious" is "how can I avoid being in this position so often". Which is what a lot of the advice here has adressed.

Clojure being lazy means that the stack traces are always "deep",

If you're primarily interested in making it easy to refractor across your code base, I don't think a dynamic language is the right choice.

Indeed, we all want to build better software faster

My main interest is building the right product, but refactoring as we learn from customers when bringing it to market and discovering the real requirements in the field is a big piece of that.

Isolation and layering help you manage the impact.

Agreed. But again, even with that the feedback loop from changing something to learning its impact in Clojure is slow because you have to have enough tests to cover it and then you have to wait for the tests to run to find out where the impact is. And then you have to dig through a stack trace if you are lucky enough to have an assertion to help you locate the error or work backwards from a test case failing to what is the path of that value to the result in the output. At that point if you are me you probably start adding :pre {(money? x)} to a bunch of functions just to get better feedback next time. And then you run tests for the namespace and the whole application. All while thinking, could I get this feedback faster?

I don't think that a lot. I think we approach both design and refactoring differently.

I feel like people is trying to arrive to a absolute conclusion about quick feedback when refactoring when I think there are a lot of tradeoffs as usual. IMHO changing types is going to be faster in a strictly typed language with a faster feedback loop, while other things like trying a refactor (which sometimes you are not sure about) to a small part of the system gets much slower in a strictly typed language because you need to fix all compilation errors before you can try your refactored sub system, just to realize that refactoring that way doesn't work and you want to try something else.

I'm enjoying all these discussions. I think everyone makes valid point. The part I can't really speak too is your custom DSL for contract definition and how it's implemented as an interpreter. But for complex domain, I normally apply Domain Driven Design in Clojure. I have a relatively trivial example here: https://github.com/didibus/clj-ddd-example Definitely I think because Clojure has no types and allows you to use maps and vectors for modeling data, people often skip modeling their data altogether. You really should model your entities and their relationships, and your operations that modify them and the invariants about correct changes, those are good things. I always have a constructor function, and I like your idea of having a predicate function, I normally just validate with spec instead, but wrapping that in a function could be practical. As someone said earlier as well, it's common to have one namespace for each entity. Another thing is, I'm not sure if you're familiar with the pattern, and can't remember the name right now, but it's basically where you always use a structure to model everything. So even instead of doing:

String userId;

UserId userId;

(ns money)

(defn make-money [amount]
  {:amount amount})

Just ran across this article today which feels relevant to this thread (since the original essay to which it refers probably aligns more with Martin's feelings): https://www.devcycle.co.uk/Clojure-the-devil-2/

Well I think this kind of thing becomes easier when you start to think of such events as not merely a side effect of your business logic, but first class things in themselves - ie model events as part of your domain model.

Agree, just wanted to avoid building a tool to represent this events and visualise them. At least on the beginning. It can be challenging itself.

Perhaps something like Elastic Search with its extensive tooling, or Grafana or the Clojure Riemann library that some people use for event processing for IT operations

My go-to is mulog for structured logging and the ELK stack or some variant for querying/visualization

I can only think of things that do this peripherally, not as a central focus 😕 Looking forward to see if someone else has a resource!

> Something that shows you what functions are available in the standard libs and how they can be used and combined? https://www.manning.com/books/clojure-the-essential-reference

there is a website where you have challenges to code small tasks without using function X. You can preview the best answers. I think this is really good place to learn how to use Clojure in 1-2 lines instead of 10. I forgot a link. I am sure someone else remember and will paste it here 🙂

4Clojure?

yes, I think it is. I have a long break with it. I don’t know better place, than this one.

It’s excellent imo, along with Clojure/Clojurescript koans, but that’s more for learning the technical philosophy of Clojure in tiny steps than writing idiomatic code.

oh after that, then probably only developing and writing code. In other word only experience. I believe there is some level, after which reading books etc. don’t help us much as doing hobby projects.

but this is my opinion

@U05476190 https://www.manning.com/books/clojure-the-essential-reference looks like it could be just what I'm looking for. It's been in development for a long time though, and there doesn't appear to be an option to reserve a hard copy. Copying in @U054W022G in case he cares for this feedback. Apologies for the spam otherwise

4clojure looks like it would be useful as well. Thanks for that @U0WL6FA77 and @U0AQ3HP9U. Definitely a compliment to what I'm looking for, but I'd also like something more opinionated that leads the learning.

I’ve come to enjoy opinionated text over the years, perhaps as I’ve become more experienced myself and better at evaluating the opinions I see 🙂

Godspeed on your journey!

I've been following the book for a long time. The last MEAP update was last year, but quoting the email: > Clojure, The Essential Reference is headed to Production where it will get a final polishing before publication. So it is definitely complete enough to serve as a good reference. If you don't mind reading it on a device, I can highly recommend it. 👍

Yes, @UVDD67FFX long story short, book development ended 1 year ago. Manning doesn't want to print it as it is too big. This looks to be connected to removing the MEAP, for some of their internal processes, they will then have to do something about who bought the book and expecting a printed copy. So they are going to leave it in MEAP forever. The original sin on my side was not to do this work as self-publishing and make it free. But I'm working to fix my mistakes, so if everything goes well, we'll see the book free in 2023.

It must have been a real labor of love @U054W022G. I for one really appreciate the book and thanks for sharing your thoughts on The REPL podcast 🙌

hey @U05476190 can you give an example of thing which you changed after reading this book? While everyone like this book, then maybe I will take a look on it. PS How is it going?

@UVDD67FFX i humbly offer up https://github.com/robert-stuttaford/bridge as an example of how one might build a complete (in that whole coherent features exist), idiomatic clojure program. comes with several dev-diary blog posts, too!

@U054W022G Good luck. That must have been quite demoralising after all the effort you went to. I got myself the PDF but I look forward to having a copy on my shelf by this time next year.

@U0509NKGK thx

@U054W022G I'm so looking forward to that, I've read a lot of it in early access and it's great.

@U054W022G you've probably already thought about this, but just in case; perhaps some sort of financial support from #C8145Q6VD may help in covering a rights purchase back from Manning?

given that you'd like to publish it free and open, i can see how that would be worthy of such support

of course it may be that the numbers don't line up at all and this is a terrible idea, but i thought i'd share it just in case 😅

@UVDD67FFX Clojure For The Brave And True from @U0AQ1R7FG was a great introduction resource for me. You can read free online or support by buying a physical or digital copy iirc. https://www.braveclojure.com/ #C0M8PCF7U

Thanks @U5RFD1733. I think that's one of the first books I tried to read on Clojure. Unfortunately I was never able to get on with the author's writing style.

The Clojure: The essential reference is fantastic, but as the name says, it is a reference book. It provides details into every function, but I would not recommend for a beginner. I think the Joy of Clojure (also by Manning) may be a better choice to learn about functional programming in Clojure.

I think “Getting Clojure” by Russ Olsen is a first good book. I just didn't think that was the intention of the OP, although I may have misunderstood.

No, you were right. I'm quite experienced with Clojure already. What I notice, though, is that more more experienced Clojure devs combine functions that I don't always know about in a way that produces comparatively simpler code. That's the skill I'm trying to develop

Found this gem today. It should have lots of utility http://jafingerhut.github.io/cheatsheet/clojuredocs/cheatsheet-tiptip-cdocs-summary.html

https://clojure.org/api/cheatsheet

if you've got stuff with spaces, why not use strings?

or use a custom tagged literal #my/keyword "hi mom" and read into a keyword

or tag #clerk/keyword "my keyword"

dang. too slow

yep, can use a tagged literal

mainly wondering about me changing print-method

regardless of via read-eval or with a custom tag

thinking long-term I want to switch to transit for which this problem doesn’t exist

so I’m mainly wondering about negative side effects (with other tooling?) from changing print-method for non-roundtrippable keywords and symbols

though I guess they can’t be read anyway so probably broken one way or another

I think it's pretty evil bold to change print-method on keywords

Writing readable edn reliably is surprisingly hard. There are 6 (at least?) different dynamic variables that can affect how data is printed. Searching on http://grep.app, I hardly see anyone explicitly set more than 2.

I think this is one of those things where, if you control both the sender and receiver of this data, then I guess you're free to bend semantics of clojure data at your will, but it's definitely non-standard and should be avoided whenever possible.

there’s no less invasive (non-global) way for me to do this that’s not a lot more code, right?

Well, if you are trying to transport e.g. a map and it contains keys with spaces in it, then I would just use strings as keys there. But I'm not sure I fully understand the problem you're trying to solve, yet.

this isn’t data that I control, it’s for clerk, I’m trying to fix an issue that if you want to show result that contains unreadable symbols or keywords I can’t get it across the wire using edn

I’m using edn in edn encoding for each result already so a single unreadable edn doesn’t infect the whole doc but it would still be nice to offer a better UX in this case

but if the advice is to move from edn to transit that’s also fine

it's true that edn is hard to write reliably

I have been advocating that we should work on the general problem of printing roundtrippable edn but haven't gotten that one on the board yet :)

there's the separate problem that the print system is global, but apps might want to have local control

yes, I do think that is also a general problem

great to hear this is something on your mind

I guess it'd be worthwhile debating on a sort of "literal" reader like Common Lisp has for symbols. e.g. a keyword with spaces looks :|like this|, but I'm not sure if any equivalent exists in Clojure yet. Nonetheless a cool feature to have

Only downside is that | characters in the middle of a symbol would "toggle off" this reader... e.g. :|this|looks|funny| would correspond to :|thisLOOKSfunny| or something like that

The | is reserved for that and I did a prototype of it back in the 1.7 era but we decided it was not (yet) worth doing

But that’s only one of many such issues

In any case, I'm glad to hear the core team is thinking about these things and being careful with any particular decision! Although it means waiting forever for new features, there's something I really like about this careful approach and brainstorming several ideas (and rejecting many of them, too).

This seems like behavior specific to deps.edn. I can't reproduce this issue with Leiningen. With that said, maybe try {:jvm-opts ["--add-modules jdk.incubator.foreign"]}? Unfortunately this means cramming modules into a single string, but otherwise I expect this to work as intended

I have the same behaviour with leiningen. A profile like:

:jdk-17 {:jvm-opts ["--add-modules" "jdk.incubator.foreign,jdk.incubator.vector"
                                 "--enable-native-access=ALL-UNNAMED"]}

-Dclerk.cache_dir=/home/carsten/.clerk/cache
-Dfile.encoding=UTF-8
-Djdk.attach.allowAttachSelf=true
--add-modules=jdk.incubator.foreign,jdk.incubator.vector
--enable-native-access=ALL-UNNAMED
-Dclojure.compile.path=/home/carsten/Dropbox/sources/tech.ml.dataset/target/classes
-Dtech.ml.dataset.version=6.103-SNAPSHOT
-Dclojure.debug=false

So again, an '=' was added

In any case, bause seem to be "valid". The jvm help says:

To specify an argument for a long option, you can use --<name>=<value> or
--<name> <value>.

Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:445).
jdk.incubator.foreign.MemoryAddress

ok, the problem is not in Clojure / Java. I'am using an embedded JVM from python, and for this the following calls are not equivalent:

javabridge.start_vm(args=["--add-modules","jdk.incubator.foreign"])
javabridge.start_vm(args=["--add-modules=jdk.incubator.foreign"])