Clojurians Log v2

This is how I stumbled upon it https://x.com/KasparsDancis/status/2107445371886539203

> The biggest worry was bundle size but it only changed by ~2%. The production build time was reduced in half. Dev build and hot reload via Vite is also several times faster (there's no longer need for shadow-cljs).

A lot of design overlap with squint/cherry btw: ESM, Closure-less, vite

is it done with AI? i am not surprised but i don't see anything indicating it.

yes, all AI

https://x.com/KasparsDancis/status/2107456879647429084

hey @k7d, we're talking about your fork!

hey

It's always crazy when somebody still uses Twitter

Like that's the CP site why are you on there

Anyways sorry didn't mean to derail

@emccue fwiw i’m asking that question myself regularly, wtf i’m doing there 😅

can we please talk about ClojureScript here? ;)

so for the context, Whimsical is large 9 year old ClojureScript (Reagent) app. For a while we have struggled with 2 things - the build times both dev and prod are slow and it doesn’t really play nice with JS ecosystem. We have couple low-level TypeScript libs internally for performance reasons and developing those alongside ClojureScript was PITA. ALso we have been bitten by extern issues many times over the years. I’ve looked at Squint and Cherry and that’s kind of what we wanted but that seemed like a massive change to existing codebase. So finally decided as experiment to fork ClojureScript and see if we can replace Closure with vite toolchain. And it worked suprisingly well. As of yesterday, it’s running whimsical app in productiojn.

Got it. I guess a lot of people are using shadow. I don't consider shadow a fork of CLJS, it re-uses most of it and puts it together in a way that's more palatable for some consumers (+ ESM output etc). Just wondering if Whimsical couldn't have done it the shadow way to get their benefits. Curious to hear about the trade-offs. I'm not judging you, but it'd be great if we can prevent fragmentation and improve collaboration (I know it's a tough process)

(afk for a bit)

As for shadow-cljs-like approach, honestly I’m not 100% sure. My biggest gripe with ClojureScript has always been Closure compiler (which shadow-cljs works around a little bit). I figured, it’s currently so core to ClojureScript is that the best approach is to just pull it out directly. Ultimately our priority was to make our codebase work better. We weren’t planning to crate a new public dialect of Clojure. Since it worked so well, I figured I’ll just share it with community as food for dicussion.

There is a more controversial angle too - last few years we’ve really started to miss static typing in our codebase. Not sure how realistic or dumb this is but one specific idea I want to explore in future is whether we can add TypeScript-based types and have Clojure compiler transpile to TS which is then validated by TypeScript compiler (and can feed back type errors to user pointing to source Clojure code).

Closure Compiler is not really core to ClojureScript, remember we can self-bootstrap. CC is only need for optimized builds. shadow-cljs is the same, so I'm not sure why you couldn't just use the shadow dev output and pass that to Vite instead?

I also don't see how you can avoid damaging the REPL experience w/ ESM? that's the reason it's optional in shadow.

We might look into trying a shadow-cljs like sidekick approach. At a quick glance there are some complexities but might be solvable: > - These emit* methods have ES module branches in the middle of their logic: :ns, :def, :set!, :invoke, :fn, :the-var, :deftype/:defrecord, :js and :if. The wrapper would save the original with get-method, call it for classic builds, and use a modified copy for ES module builds. > - The same applies to analyze-deps (skipping goog.* analysis) and parse-require-spec (recording npm requires).

@dnolen not a lot of milage on it to judge what issues it has but we do have working repl https://github.com/WhimsicalCode/clojurific

ESM modules are effectively closed, you can reload a whole module, but as far as I know you cannot redef an single var etc.

> Imports are read-only, but they’re live bindings, not copies. When a module reassigns one of its own top-level variables, every module > that imports it sees the new value right away. So Clojurific has each namespace’s module export a setter that runs inside the module

also imports are captured

// out/my/util.js (simplified) var my$util$format = (function my$util$format(name){ ... }); export function $$set(name, v) { switch (name) { case "format": return (my$util$format = v); } throw new Error("No var my.util/" + name); } export { my$util$format as format }; // out/my/app.js import * as my$util from "./util.js"; ... my$util.format(name) ... // reads the live binding on every call When you evaluate (defn format [name] ...) in my.util at the REPL, the form is compiled to a function that the browser runtime calls. The def turns into $$set(“format”, newFn). Every caller, including http://my.app, which imported it long ago, gets the new definition on its next call. Nothing is reloaded. The same setters are how set!, binding and with-redefs work on another namespace’s vars.

also: - Hot reloads keep old instances in sync. A hot reload runs a new copy of the module, but modules that imported the old copy still hold its bindings. The new copy pushes its values into every earlier copy through their setters, so old and new importers all see the same definitions. A later REPL def goes through every copy’s setter. - New vars live in an overlay. A var defined only at the REPL, one the module never had, can’t be added to a compiled module. It goes into a per-namespace overlay that REPL-compiled code reads. That’s enough for interactive use: the module’s existing code can’t refer to a var it was compiled without anyway. Once the file is saved, the namespace recompiles and the var becomes a real binding. - It’s development only. The REPL wiring (each module registering with the runtime and its setters) is only emitted for watch builds. Production bundles don’t have it. So redefining a single var works much as it does with goog-style namespaces: the next call picks up the new definition. The difference is that the assignment happens inside the owning module instead of on a global namespace object.

yes live bindings help, but I forget what the cost is, so new stuff to consider there, but again since ESM isn't really a global thing a bunch of other expectations are going to subtly different.

i.e. I make a new def, and then I can in a REPL session just get at it because I loaded that ns at some point etc.

yeah there are likely subtle issues that will showup over time. that being said we haven’t been using REPL for app dev much lately so it wasn’t a high priority. Fast and NPM-library-compatible hot-reload in dev was much more important (+ properly supporting JS/TS library source maps).

sure I'm just pointing that these are very non-obvious things about ESM. It sounds like a good idea only if you're going drop a bunch of things 🙂

there are always tradeoffs 🙂

I just mean in communicating in the project if it's something you want other people to consider. The ESM output breaks a bunch of REPL corner cases because ESM and Clojure namespaces are two incompatible systems.

I also don't see how you can avoid damaging the REPL experience w/ ESM? that's the reason it's optional in shadow.

I use a similar model as CLJS (nested objects) in REPL mode along with emitting an ES module. Dynamic vars stay mutable cells with getter + setter in production to work around ES modules being immutable. set! on a non-dynamic var doesn't work

yeah, you need the REPL to recover the expected behavior. Yeah and special casing dynamic vars makes sense.

and I suppose something different for with-redefs too? or that's just not supported.

not supported, only dynamic vars supported in this model. with-redefs or binding are just the same here.

$ ./node_cli.js --show --no-run -e '(def x 1) (def ^:dynamic *y* 2) (+ x *y*)'
import * as squint_core from 'squint-cljs/core.js';
var x = 1;
var _STAR_y_STAR_ = ({val: 2});
(x + _STAR_y_STAR_.val);

export { x, _STAR_y_STAR_ }

I'm a bit out of touch - what exactly are the restrictions of ESM modules?

One thing (the most important restriction for CLJS) is that you don't get to mutate exported values with = from the outside

Clojure's namespace system is really a just a convenient global organization mechanism, ESM is not that.

to be clear ESM was created by ex-Racket people and Racket also dropped the Common Lisp / Scheme global we're all adults here approach to software development.

That is one thing JVM modules do allow, but only if you have explicit permissions for doing so.

In my understanding, ESM is easier to optimize because of the immutable restriction, you can just look at the imports basically and strip out any untouched functions (papering over some details here) So it's easier for dumber optimizers than Google Closure ;)

https://dev.to/tassiofront/barrel-files-and-why-you-should-stop-using-them-now-bc4

There's lots of articles about this particular problem w/ ESM

something that I think would be more useful than dealing w/ ESM, would be something that lowers all your deps to Closure format, this is what Google does, it used to supported via tsickle but they dropped external support. Internally I'm pretty sure they do this.

then you can apply advanced optimizations to your deps, those are the ones that need it because that's where the bloat is coming from.

Well if you remember from the fish place @dnolen, my special interest at the moment is "strong runtime protected modules" - so i'll start digging into this

yes, I've been waiting for JS minification to best Closure for 17 years. ESM does not fix anything at all. If your transient deps use barrel files, nothing is going to get removed.

So we are getting quite a bit of value from the advanced optimizations. We have gotten a fair amount of breakages over the years from advanced, since it doesn't click for people/agents, but we can probably fix that.

Sometimes people also add npm dependencies for things that seem like aren't really doing anything. Not quite left-pad level, but some of them might be close

But yea we are also looking into datastar type approaches which will let us cut down a lot of the client logic

re: ESM optional output from ClojureScript, I'm not against it but I do not think it is a very small project to handle all the cases (including warning about unsupported forms). So it's never been very appealing.

I think many people would value the easier interop over the minimization losses. Its a tradeoff either way.

On the other hand, a tool to look at your npm requires and auto lowers them to GCL - now that's something that's only win. You can use everything, all the REPL stuff works, less extern problems, and smaller bundles.

the problem w/ this idea is that NPM stuff is so chaotic and every project is some cross product of bad ideas of build requirements.

@isak fwiw I suspect it would be smaller than that if you compile it with our fork. It does a bit more than just simple+vite, f.e keyword hoisting and namespaced symbol shortening. In our case the difference was about 2%. Our codebase is about 2x yours in size.

anyway, @dnolen just wanted to use an opportunity and say that i’m deeply grateful to you and the rest of the team for ClojureScript and Clojure. I don’t think I could have made Whimsical work without it

Whimsical is awesome btw! Used it some years ago w/ the team at Vouch. It's definitely an app that's hard to imagine w/o heavy client side lifting.

Good to finally see a pure ESM port. Been meaning to explore that myself for quite some time. sooo many questions 😛 , but given that AI wrote it, they are all answered 😉

> Can't blame them for doing it, but mixed feelings cmon, leave a slice of Clojure dialects to the rest of the world 😁 else it's not a real Lisp curse

that reminds me - where did the clojure standard work that was going on last year end up?

i forget what that was called, but i remember there were discussions about defining what clojure actually is given that we had so many dialects popping up

what I'd really like to see is a build report comparison, but not sure they exist for vite? my instinct says the size contributed by CLJS went up, but the size of the JS went down. thats why they are so "close".

nvm Hello world: 177 KB / 35 KB gzipped (Closure advanced: 110 KB / 23 KB). sort of confirms that

> I guess that's what we can expect in this AI era now. Not sure how I feel about it. It's cool that you can do these things, but we'll also see fragmentation and less collaboration. Can't blame them for doing it, but mixed feelings. I dare to object: • the Lisp curse was a thing long before the current AI wave and Clojure itself (and it's a lot about fragmentation, etc) • c'mon. Leave a slice of Clojure dialects-fun to the rest of the world 😁 But in general yes, it is what it is but I somewhat still prefer everyone crafting their own Lisp than the tedious rewritten to Rust - coz is fast - with AI! Release date unknown, but so cool!

vite does have built reports using a rollup plugin

or rolldown or whatever they call it

nice

Yea it seems very good based on the stats, haven't tried the app yet. Interestingly the un-gzipped size is 13% larger than advanced.

> I've been waiting for JS minification to best Closure for 17 years that didn't age well 😅

@dnolen are you a heavy cljs REPL user? I always struggled to set it up comfortably. I hit lots of quirks.

I just use the plain REPL same as how I use Clojure. I did this even when I was using IntelliJ / Cursive, I never had problems. I did use Figwheel a lot in the past as well, and a bit of Shadow. The plain REPL has nothing that needs understanding in my experience.

re: Closure vs. JS tools, I never seen any real good comparison here, i.e. someone actually converts a la Google style. my understanding is that Google could not get acceptable DCE w/ JS tools, thus they convert to Google Closure before running advanced optimizations to eliminate the ESM semantics.

i.e. comparing Closure vs. Vite on moment.js is meaningless. Using the GCL time facilities of course is better because it eliminates everything w/o futzing w/ imports

once your project becomes large (and you did not do something up front), I'm not sure you can really fix anything because too much work - for this it might very well be that the difference between Vite / Closure becomes uninteresting. A few percentage points in either direction.