Clojurians Log v2

instead of posting here, the best place to discuss and propose changes to clojure.core is at http://ask.clojure.org

you may also be interested in https://aphyr.com/posts/360-loopr-a-loop-reduction-macro-for-clojure

> A generic client like the AWS-API would be enough this is exactly what I wanted too, aws-api is close to perfect for this kind of thing. I think I'll try to release what I came up with for openAI soon. The tricky part about it is http kind of is not fun to use for services APIs IMO, there's tons of arbitrary choices the api designer can make (e.g. should this arg go in query params or body, how do we use http methods, headers, etc) so you have to learn and remember so much "incidental complexity" every time you want to use a thing. But the more you try to abstract that away so you can think about it less, the more likely you are to settle on an abstraction that is leaky/brittle/relying on vague conventions some particular api's use anyway. (for instance I used openAPI's operationId field as something like aws-api's :op, which is nicer than having to remember and type endpoint+method, but operationId is optional so relying on that won't let you work with every openAPI spec. I also tried to just merge all args such that I can stick all query/body/post params into one arg map, which worked for me, but relies on the fact that these names dont clash, when that is not a requirement of openAPI. So you end up having to make a lot of these precarious choices if you want a nice to use client. I think the specification format used to drive aws-api (which is I think related to, but not exactly, Smithy) is a bit nicer because it's more restrictive. Out of curiosity what api are you looking to make a client for? I got deep into the openAI spec and I'm curious if other ones have the same quirks I hit. > BTW, I've seen that you have contributed to coffi. What are your experience with it. I'm not an expert, my contribution was very small. I would say the main cool thing to check out is the java.foreign api (aka project panama) which coffi provides a nice light wrapper for but doesn't really introduce any new concepts. If I wanted to wrap just a couple C functions, or work with MemoryLayouts, I would probably start with coffi, if you want to generate a full wrapper I would look at jextract or https://github.com/phronmophobic/clong (which I had got working with its coffi backend but I'm not sure if it's officially supported). Might be useful to check out #coffi if you haven't yet

I want to build a client for the IBM Quantum/Qiskit REST API to be able to submit quantum circuits to their QPUs with https://github.com/lsolbach/qclojure. I've implemented an Amazon Braket adapter with the AWS API (yet to be tested) and I'm reseaching the options to build adapters for other providers. The ArrayFire adaption would be for the local simulation.,

I haven't used it extensively but there's https://github.com/oliyh/martian You can see some realistic usage of it here: https://github.com/wkok/openai-clojure/blob/main/src/wkok/openai_clojure/openai.clj I recently opted to roll my own instead. With just the yaml spec and a http client of your choice it's not that bad getting started with building your own request/response transformers to handle everything you need. I suspect that once an API gets suitably complex you'll need to add a lot of custom logic anyway, either as martian interceptors or whatever. (eg for stuff like SSE, blob data, errors in the particular openAPI specification, which I seem to always hit, etc)

A generic client like the AWS-API would be enough, Most of the logic is implemented elsewhere anyway, it would be a simple adapter. I've started to implement something myself, too. But if there is some library or tool already, I'm happy to take a look. So thanks for the tip.

BTW, I've seen that you have contributed to coffi. What are your experience with it. I'm thinking about using it to call ArrayFire for complex linear algebra on the CPU and GPU. That's missing in neanderthal.

No language will prevent you from writing bugs, not rust https://github.com/rust-fuzz/trophy-case (and not even some stricter language like idris). The hardest problem, imo, is where your program meets the real world or other programs not under your control, there's 40+ years of layered dependencies that our programs balance on. Yes, the risk of type mismatch errors in a dynamic language is bigger, but it's a relatively easy thing to catch and fix compared to other mistakes, and not one made frequently in the first place (you're unlikely to pass 50 to a name property). What matters the most after a refactoring is whether your program still works, and there's no better substitute than testing. Types do help tooling (IDE's, linters, etc..) a bit, but the cost is rigidity to changes both because of the syntax and also rigid designs caused by how one has to translate into categories and patterns ehttps://clojurians.slack.com/archives/C03RZGPG3/p1753966664816079

Are you looking to expand your mind, or are you looking to further your career in the short term? If the latter, I think you should look at your local job market and take that into account in your decision-making.

> You may want to talk to @jyn514 , she’s quite a Rust expert who also picked up Clojure! i have not worked on large clojure projects so i only have so much to talk about there. i will say that rust is extremely good at letting you refactor large established codebases, in a way that would be insane in any other language (https://matklad.github.io/2020/07/15/two-beautiful-programs.html has two small examples). rust doesn't "scale down" very well though; it's really a pain to use it for advent of code, and even for medium-sized personal projects i find it slow to prototype with. if the thing you miss from rust is the compiler giving you guidance, i second malli, i think for the scale of 1-10 person projects it would work quite well. i don't think it would scale to 100-1000 person projects, but i don't know any clojure projects that large so i'm extrapolating. > Static types are nice in helping with code readability, refactoring and IDE auto-complete. Those are the things you trade away unfortunately. So normally having doc-strings and good names help with quickly knowing the type of something for readability (can also use lightweight spec and similar). For refactoring it's too bad, type info cannot help the refactor tool, and breaking a type won't show you precisely where it was used, you need to spend a bit more time running tests and manually tracking breakage 👍 seconding this, Rust-Analyzer is really quite good, not as good as Intellj for Java but nothing is

> rust is extremely good at letting you refactor large established codebases, in a way that would be insane in any other language if this interests you, ask me about refactoring rustc's name resolution to be incremental

Sean wrote: > We use Spec heavily, but not as a "type system". I've written about it on my blog http://corfield.org (on my phone at the pub otherwise I'd link you to the article). here's a link https://corfield.org/blog/2019/09/13/using-spec/

I'll add my two cents here. I've worked on Clojure projects of all shapes and sizes (small prototypes up to hundreds of engineers and many apps/services/lambdas). I want to echo what @seancorfield said. He's right on the money. > One of the things with Clojure is that functions tend to be permissive on what they accept so "refactoring" is often a lot more localized because you're often not changing signatures (and types) so you don't get the same sort of cascade that you can see in statically typed languages. > 100% correct. You update a producer of data somewhere (e.g. adding a key to a map) and then you update the consumer elsewhere. You don't have a verbose type changing cascade on the intermediate data objects, which always bothered me in Java.

Also, when dealing with large, distributed systems, particularly involving message queues or brokers, I've found that guarding the inputs and outputs of the queue with Malli or something similar really allows Clojure to scale with little friction.

Now on the flip side, do I spend more time using "find and replace" instead of an automatic refactor? Sure! But I've found it's negligible to my productivity.

i will say that rust is extremely good at letting you refactor large established codebases, in a way that would be insane in any other language

@jyn514 And that's a really good point. Very rarely I hit a situation where my Java expertise comes into play. And I can just use Java interop to solve my issue. The last one like that was, in fact, a thread safety issue.

> there are other weirder ones for example you can say (binding [*cx* (f)] ; ... in such a way that you can statically guarantee that *cx* cannot escape or be returned from that binding call

So maybe my love of Clojure is in part due to my Java background, and someone new entering the space should be aware of that as a consideration.

> Now on the flip side, do I spend more time using "find and replace" instead of an automatic refactor? Sure! But I've found it's negligible to my productivity. Between LSP's rename support and Copilot/Claude/etc, I find most refactorings can be mostly automated...

Personally I don't use any AI tool for direct code editing/generation so I don't have any experience with that. But your callout about LSP is on point. Renaming symbols is very easy and works great (Sean, I think we both use VSCode/Calva, correct?)

Correct me if I'm wrong, but I don't think there's any support for renaming keywords. So I usually default to "find + replace" for that task.

Yup, I'm on VS Code + Calva these days (after decades on and off with Emacs). Plus Portal for data visualization and Joyride for scripting VS Code via cljs 🙂

clojure's LSP is reasonably good but really can't handle unbalanced parens

Ah I need to try Joyride. I would love to script VSCode. That'd be a great QoL improvement.

See my vscode-calva-setup repo for the Joyride scripts I use 🙂

I work in a large Clojure code base and it’s great!

I also work on a Clojure code base, and it's nice. Usually the type checking doesn't cause a problem for us as far as we've been able to tell. We use things like plumatic/schema and metosin/malli to validate incoming request bodies, which allows us to check for errors without having strict type checking.

One thing I like about Clojure, is that it's possible to learn it in a piecemeal fashion, so that the up-front cost for adopting Clojure is fairly low, and you can learn more as you go. Of course, once you know a few programming languages, it gets easier to pick up new ones. You can always start with the one you prefer then spend some time learning the other.

At work we have 150k lines of Clojure and don't find the lack of types to be a problem. One of the things with Clojure is that functions tend to be permissive on what they accept so "refactoring" is often a lot more localized because you're often not changing signatures (and types) so you don't get the same sort of cascade that you can see in statically typed languages. My opinion is that the choice between static and dynamic typing is purely subjective. Some folks like types, some don't.

We use Spec heavily, but not as a "type system". I've written about it on my blog http://corfield.org (on my phone at the pub otherwise I'd link you to the article).

I'd say no, malli/spec don't really help for the problem you are talking about.

Static types are nice in helping with code readability, refactoring and IDE auto-complete. Those are the things you trade away unfortunately. So normally having doc-strings and good names help with quickly knowing the type of something for readability (can also use lightweight spec and similar). For refactoring it's too bad, type info cannot help the refactor tool, and breaking a type won't show you precisely where it was used, you need to spend a bit more time running tests and manually tracking breakage. For auto-complete, leveraging namespaces can help, or having type prefixes to filter down the candidates, as the auto-complete cannot use type info to do so. But this is the "cons" of Clojure to be honest.

If you don't find the "pros" outweigh these "cons", you probably won't be happy with choosing Clojure. The pros are worth mentioning a little bit, the interactive development along with the immutable and functional nature, does help with refactoring or general coding in other ways. Less code is needed for the same functionality, which reduces the overall code surface and means maybe you need to refactor less often or just there's less code to update when you need to refactor. The dynamicity and overall idioms help create decoupled isolated code units. That also limits the instances where you might need to refactor. And backward compatible changes can be easier to maintain, avoiding breakage and the need to refactor. And then there's everything else, the syntax, abstractions, the ease of meta-programming, etc.

I have to refactor a lot less in Clojure, I still haven't figured out exactly why. But I believe the interactive development means you find a good factoring right away, you iterate so fast you've already iterated on how you want to factor the code, so you won't have to do it later on. Even when refactoring, how do you know what to refactor too? The interactive nature of Clojure lets you experiment way faster and discover the correct code structure much faster I think.

Yeah, I think it's all very subjective. I like Rust. I didn't think I'd want to use it for my job tho'. If it wasn't Clojure, I might go for Kotlin.

You may want to talk to @jyn514 , she’s quite a Rust expert who also picked up Clojure!

I struggled with this question until I got comfortable leaning on namespaced keywords, as encouraged by Datomic. My previous language of love was Haskell, which has a similarly powerful type system. Instead of type definitions, I now start with the Datomic schema and example values. One thing I've gained is less friction between in-memory data and the stuff in the database. Typed Rust SQL solutions like sqlx is going to get you a part of that value, but you still have to decide how much data you want to pull out. Datomic's entity API lets you stop worrying about that, and treat information from the database as a map (struct) of namespace qualified keys ("types") to data. With something like postgres, you have to join and juggle keys, Datomic can smooth over that. --- So, where you'd type your rust values, use maps of namespaced keywords, and treat those individual keywords as types for specific attributes. --- This gives you two more advantages over typed Haskell / Rust code: 1. Since you typed your attributes and not your objects, you can re-use typed attributes in different places. Every place you talk about a person's email, use :person/email. 2. Instant test / REPL feedback. You mentioned Common Lisp, so essentially, yeah, you get to keep that nice, fast CL feedback loop.

There are sort of two phases to consider, generating examples and testing them, and once an example has failed shrinking and retesting

So an initial question might be which of those phases is it churning in

So you can add a println on failure or something in your spec to see

An alternative hypothesis - I was just lucky when I tried dozens of times to 60 and unlucky when I tried a few times with 70 to get inputs that made the function being tested behave abnormally. I'll see if that's the case first since at least it would mean that it's my own code that's at fault.

That would be one of the possibilities leading to churning in the first phase

Huh, the default generator for inst? is weird. The vast majority of samples fluctuate around 1970 and most of the other values skyrocket to year 8312240 or whatever.

https://clojurians.slack.com/archives/C03RZGPG3/p1670964591495919?thread_ts=1670961891.796179&cid=C03RZGPG3

Ah yeah, I remember. :) Although there the main problem was with how insts are printed. And apparently s/inst-in also isn't great...