you'd need to generate overloads for the method

where method(x, y) calls method(x, y, z) with z = null, and handle the null appropriately

so the contract would be your generated "protobuf" code can add new methods, but never change existing ones' signatures

Ya exactly. So our internal "protobuf". If you add a field to an API. It will just generate new method with three args now. So it's a breaking change 😛

that sounds like hell tbh 😅

but that existing method(x, y) should still exist!

so existing callers won't break

No it doesn't. The Java code generator just looks at the new API model that says it now has x, y and z and recreates the class which now has a method of three arguments lol

yeah so what I mean is you'd have to change the code generator to not do that

or rather to make it generate a 1-arg constructor, 2-arg constructor, etc. up to N for a message that has N fields

Ya, it actually has a way to do that. You need to annotate the fields as "added". And for those it overloads, and for the others they are on all overload. But most engineers don't realize that, and then just add a field thinking it's not a breaking change and break all clients 😛

So the model would say: method: • x • y • (added) z And now it would generate: method(x, y) method(x, y, z)

Which is also funny, cause you could be looking at a model that says (added) z, where z was added like 6 years ago lol

the good old Bad Default

you could solve it by updating the code generator!

if that's at all feasible

I feel the Rich talk "Maybe Not" was basically talking about this problem lol.

if I were to design something like that, I'd almost want the reverse effect, breaking should be opt-in

protobuf has its fair share of pointy edges, but it deals with this problem well at least

well, as long as you don't ever used required fields in protobuf v2 😬

(they removed that feature in v3)

from the official docs 😂

anyway @U0K064KQV, where did you find studies about the correlation between programming language and "correctness"? That sounds really interesting; googling unfortunately didn't yield much

it's weird to me that for the trillions of dollars spent on programming computers, we've studied how to actually do it properly so little

CS is a nascent field

we're still on the "alchemy" phase :rolling_on_the_floor_laughing:

Well, that's not entirely true -- there have been several "do it properly" projects but they're also not terribly practical for building stuff 🙂

MALPAS and Z for example.

There's a lot of stuff in modern languages you essentially have to eliminate in order to "prove" anything useful about the code.

And nearly all our external representations are lossy: JSON, databases. That kills a lot of type information (explicit or implicitly derived).

I went in a big rabbit whole a while back. There were some very crappy ones that just like had students or a very small number of people implement the same "super small program" in a few language, and assess how fast/correct the result was. But the good one was this one: https://dl.acm.org/doi/pdf/10.1145/3126905 Which someone else actually reproduced here: https://arxiv.org/pdf/1901.10220 The first one showed Clojure and Haskell as the most correct languages in a statistically significant way. While the reproduced one also found Clojure as the most correct in a statically significant way, but caveats that the practical effect is so small, that it mostly doesn't matter.

I guess what I mean by "we've studied how to actually do it properly so little" is not that we aren't researching cutting edge things in academia, but rather doing boring, large scale studies on what actual practices, languages, tools, etc. lead to better productivity, fewer bugs etc.

everything I've seen to that end has left a lot to be desired, but I'm probably underinformed in this area

There have been a lot of studies -- but they don't actually deduce anything useful.

I guess that is perhaps the best argument to why software engineering is really engineering: we've failed at turning it into a science 😂

programming is maths, but programmers aren't mathematicians

(pardon my usage of "programming" as opposed to "SW engineering")

Back in the '80s (I think) there was a big push for static analysis tools and a lot of studies done to try to show that they were more effective than dynamic testing. My career in the early '90s was based off that research (I wrote static analysis tools for C and C++ and did a lot of software metric calculations). But there have been so many studies that deduced conflicting results. The studies were usually flawed -- or found statistically significant results that were too small to be worth applying in the real world 😐

Some programmers are mathematicians. It used to be more common than it is now. When I went through university and into programming, it was rare to find non-mathematicians.

yeah but now it's much less the case

Perhaps to our detriment?

(that's somewhat tongue-in-cheek, in case anyone is wondering... but basic understanding of Big O notation feels like a "must have" for programmers, even today)

I do wonder where the future of programming will take us. It does seem like the greater world has finally caught on to some of the core tenets of functional programming, namely that mutation is bad and pure functions are good. But it took years of those ideas filtering through the industry for them to take hold, not studies proving that they were superior

the main problem there IMO is inertia

if the "wrong way" has better tooling, is more immediately accessible, is taught in school, is used at work, yadda yadda it'll inevitably take time for things to change

Consider that Java and Go were specifically designed to allow "mediocre" programmers to be successful -- because we need a lot more programmers than we have (or at least we need more in the pipeline, as software spills into everything).

it seems odd though that tools for "mediocre" programmers shouldn't be better than ones for experts!

[cont] there kinda is a nod to that concept in the design of Clojure itself. It's built on top of Java because even if Java (especially back then) was a completely different world, if it had to start off from nothing it would've likely been dead on arrival

My friend recently did a little side project (built a small fitness tracker web app) fully using prompting and Claude. That might be the future of programming 😝

> it seems odd though that tools for "mediocre" programmers shouldn't be better than ones for experts! Not sure I follow what you're saying there @U076H6ZE91A?

I mean that in a just world, tools for "mediocre" programmers would be nicer, easier to use, less error prone, etc. than ones for experts

not the opposite

Java is definitely less error prone than what came before it

I would say that mainstream language tooling generally is better than tooling for more esoteric languages tho'...

I suppose it comes from the tension between writing one page of code and writing a whole fleet of apps, what makes one easy maybe inherently makes the other harder

yeah that's the point, the ecosystem of a language is important in practice

The ecosystem and tooling around Java is better than around any other language I can think of... we may not like Java-the-language, but the tooling support is spectacularly helpful.

Honestly Java isn't a bad a language compared to its contemporaries. Hell, if you look at what was around in the 70s it becomes very easy to see why C is so widespread haha

Scheme has entered the chat

I was on the ANSI C++ committee for eight years and I thought Java was a wonderful breath of fresh air when it appeared! 🙂

compared to what C++ dialect? 😜

Heh, yeah, exactly... I wrote a lot of HP-dialect C++ in the early '90s...

(I'm off to bed but these have been interesting threads and I'll catch up with new posts in the morning)

I would say Golang in particular is optimized for your successor, not yourself. That is to say, the language maintains a simple syntax and encourages a style of programming where everyone uses the exact same primitives. This dampens the learning curve for newcomers to a team. Contrast this to e.g. Perl, where there are so many ways to do one thing that everyone writes in their own “dialect” of Perl. You also see this reflected in the tooling, somewhat. Just about every Golang project I’ve worked with can be built with a single go build command. Everyone uses go fmt as their formatter (so no nitpicks on import orders, indentation, etc.) The standard library has just about everything one would want to build a web backend. The only thing that’s missing for me are Swagger docs and an OAuth client. Meanwhile in other programming languages, there is a lot more variety in tooling, writing style, and so on. Every Clojure codebase I’ve worked with professionally hacked together its own build system and mini framework; sometimes you could tell who authored a certain piece of code just based off which obscure clojure.core functions were used, etc. This isn’t necessarily a bad thing; this just shows the expressivity of Clojure compared to Golang. I definitely prefer expressive languages, but I must concede that Golang optimizes well for the scenario of “company that wants people quickly onboarded and maintaining a codebase”. One thing I’ll note: Common Lisp, Clojure, and Golang are the only languages I know where one can jump-to-source all the way down to compiler internals and still end up with code that looks like something you could write yourself. Other languages, like Python and TypeScript, have this sort of “magical barrier” where users can’t easily inspect the implementation of their compiler/runtime, nor are users encouraged to do so. And the code found in the implementations looks completely different than the code found in the average application. This sounds like a niche use case, but it helps a ton in understanding the language. The fact I can understand how Clojure(Script) works by just jumping to source on primitives is something I take for granted.

I think Go is very well designed language in terms of meeting the goals they set out to achieve. I don't particularly love the resulting language myself but I've got a lot of respect for it. I do wonder though if there is inherent tension between consistency and expressiveness, or if it's more a matter of top down style "this is the correct way to do X" type commandments from the language creators, vs. practices being grown organically by the community. E.g. if deps.edn had existed from the start, we probably wouldn't have leiningen or boot. That's not related to how expressive clojure is. If only Rich had handed down a few extra commandments about how to build a web app, perhaps things would be a little more consistent 🙂

have you heard of the curse of Lisp?

no I've not

try googling it, the document is pretty old and feels a touch cult-like, but the main takeaway (for the purpose of cooperation, there is such a thing as too much expressiveness) is convincing

Clojure, much like Common Lisp, can pretty much bend to the programmer's will - the only substantial limitation vs CL being the lack of true reader macros - with enough macro-foo. For a very simple example, you can fairly easily implement Java style inheritance and it ends up feeling just as if it was there from the beginning. For a less trivial example, with enough effort you can slap static typing on your Clojure without ever leaving the repl! This is amazing for small groups because you can basically build your own thing, but without adequate discipline it's not so great for big projects where parts need to interact with each other. That said Clojure has this problem a bit less than the other Lisps, IMO because it comes with a strong opinion about its paradigm baked in and because macros are less prevalent than in CL.

I think for me that is what undercuts some of the “lisp is so great!” arguments because it goes “code is data! that means you can manipulate it easily! Oh but actually don’t do that unless you really need to“

no free lunches

it feels almost like what happens with machine learning/AI. Bespoke solutions can achieve a lot with very little, but there is a point where your best bet is to take the simple solution and tackle the problem with sheet scale

> “code is data! that means you can manipulate it easily! Oh but actually don’t do that unless you really need to“ even though i don't macro often, i'm glad to have the option — especially after seeing how a few non-lisps walk backwards throwing kludges over their shoulder into having something like macros but much less pleasant/powerful

I’m just too used to life without macros I suppose, so I don’t really feel like I’m missing out on anything with them yet.

might be illuminating to write one based on assert to concisely insist on some value else throw an illegalargumentexception 🙂 all the benefits of assert you were looking for in the other thread without the drawbacks of AssertionError

if you're working with generated code, that's more or less a macro in disguise

ha, very true

alright here it is, baby's first macro:

(defmacro check-arg [expression]
  `(when-not ~expression
     (throw (IllegalArgumentException.
             (str "Check failed: " (quote ~expression))))))

There is a couple issues there mostly related to how some seqs str themselves, but that's indeed more or less the gist of assert (sans the "disabled in prod") bit, and with a more relevant exception type.

it's also a good macro to outline what they offer over your old function: • Access to compile-time context: without macros or a special language construct, there's no accessing the unevaluated form to print it. This is truly something unique to compile-time metaprogramming like macros. • custom control flow: functions always evaluate their arguments before being called, which macros aren't forced to. This technically is doable without macros by passing a thunk which evaluates to the value rather than the value itself. This is more of a tradeoff: if you're writing something inline, passing a lambda in is more cumbersome and might not be as performant; however, macros aren't a runtime construct and therefore can't be passed around as values. Compare (for [x stuff] (something x y)) and (map #(something % y) stuff)

And of course, macros can do code generation. If you don't expand in place and instead dump the generated code into a file, that's a classic code generator; and you aren't limited to the contents of the macro invocation form either, if you wanted to you could read files, fetch stuff from the internet, launch your stash of nuclear missiles to end mankind before we're all exterminated by killer robots... (not necessarily sorted by how bad the idea is). However, macros are embedded in the language and its compilation "pipeline", and can generate code in the same file.

I don't know about Go being anything "easier to onboard" and all that. People like to believe things like that from impression or simple arguments, like it has few high level constructs so everything is just raw if and for. But just like no one believes Clojure actually came out as the most correct language and still statistically significant in the large analysis of GitHub projects, people won't want to believe Go most likely does nothing real to help onboard your junior programmers.

Go enforces the same style everywhere

I've got friends at Google, and Go doesn't seem to be any kind of magic bullet, you hear the same level of issues and troubles that you do anywhere for any large/legacy code bases.

I also don't (commonly) rely on macros I write myself - but I'm super happy they are in the language. That means libraries can rely on macros where suitable. Consider hiccup.core/html - it converts data to a html string, but the fact that it's a macro means it can prepare some of the work once (compile time) and save time when it's used.

I guess I'm challenging how much "same style" matters that much to ramp up someone. I can see using the same libraries/overall code structure helping. And maybe Go people tend to all stick to the same overall code structure and only use the standard libs. So that can help a little. But does it result in faster delivery for projects? In quicker bug fixes? In better developer retention and engagement? Hard to say.

Mostly because, it makes an assumption that the challenging part of programming is just stylistic or familiarity. Where I feel that's the least of the problem. Maybe I drank too much Clojure koolaid haha. But I think the hard part is figuring out how to tame complexity, keep things simple and decomplected, manage accrued behavior over time, limit redundancy, maximize reuse, and so on.

Jruby's memory usage feels accurate (a team I was once on inherited a jruby app. it was... not fun)

could be interesting to do the same tests under clojure to see the footprint

Common Lisp ranks quite well also. Interesting

SBCL is a great runtime, as CLers will loudly proclaim at every opportunity

Haskell is pretty consistent, neato.

The fact TypeScript compiles to essentially JavaScript without type info yet consumes 5x more energy makes the results of this paper dubious to me

Before someone pretends this is including the compiler's energy consumption: take a close look at where C++ and Rust lie 🙂

Well, to me, the first red flag was the entry of "Lisp" given that there is no such thing as a language called "Lisp". LISP 1.5 exists but its pretty old and I'm pretty sure that's not the "Lisp" this paper's referring to. Maybe it's referring to Interlisp or Maclisp or even Scheme? Who knows... Also this paper seems to do what annoys me the most about language discussions: assuming "language" and "language implementation" are identical. e.g. if you assume Lisp is "Common Lisp", which implementation? if SBCL is chosen, then one is ignoring the fast compile and startup time of other implementations, like CCL and ECL. If one uses ABCL, then they can get really bad results

They took the implementations from the benchmark game. That's probably sbcl

"Probably" SBCL, "probably" ECL for the SIMD heavy algorithms (I'd expect GCC and Clang to outperform SBCL on any vectorization). Still vague :)

And the Typescript vs JavaScript difference is comically wide

Not vague, just going in memory, which was accurate, go read it yourself https://sschakraborty.github.io/benchmark/lisp.html

One could argue that Assembler is a language and expect it to be a bit better than C.

Sysdig is close

Thanks, haven't heard of it before. But I wouldn't call it close to what I have in mind. It does both too much (whole machine monitoring) and too little (still a TUI, seemingly with no single-panel view of forks and overlapping syscalls). A workflow in strace with a very high "pain times frequency" number: 1. Notice that some complex program doesn't work and says "Oops, I crashed" in the terminal 2. Start it with strace -s 1000 -f ... 3. Find where it prints out "Oops, I crashed" 4. Follow the chain of syscalls across all the forks till you find a potential root cause, while keeping in mind that traces can be out of order due to multithreading A GUI with a vertical Gantt Chart would be helpful here. Somewhat similar to working with git log --all --parents in a messy repo as compared to using a Git GUI that can sensibly display branches.

Agreed, most tools for that are "some assembly required". You want something like Wireshark for strace but there doesn't seem to be such a thing.

Yeah, Wireshark is great. :)

Found https://kernelshark.org via https://jvns.ca/blog/2017/07/05/linux-tracing-systems/.

Ah yes, I recall seeing it years ago. Unfortunately, at least as far as I'm aware, trace-cmd cannot capture function arguments.