Hi! I want to discuss how to use Clerk to gain visibility into stateful, external processes (and whether it’s a good idea). I feel like what I’m asking for is a bit muddy. But I’d rather ask possibly vague questions than hold my silence.
PROBLEM STATEMENT: MENTAL MODEL FOR STATEFUL RUNNING PROCESSES & CLERK
I’m torn about something.
1. I find Clerk to be very effective to work with deep, composite values. For example, using the table viewer to explore what a database really contains.
a. I recently used Clerk as my “database browser” in a project where I previously would have used https://www.postgresql.org/docs/current/app-psql.html + https://tableplus.com/. I liked that I gained control (Clojure at my fingertips), and explanatory power (previous “what’s in the database?” questions were answered in notebooks).
2. I struggle with how I should architect my code to play nicely with Clerk when I have an external stateful resource.
a. A recent struggle example of this was me learning to use Playwright from Clojure with java interop. At first, I tried doing it with Clerk. Then I realized that i had to “start a playwright”, then “start a browser”, then “create a page”. All actions with side effects, and return values I needed to keep track of. I ended up stopping Clerk and continuing in a normal REPL.
i. Just a normal REPL felt quite great here, in contrast to a compile-rerun-see-what happens cycle, asking 3 browsers to all show one page, controlled from the REPL!
ii. The current state of my code a mess of functions and defs that have to be run in a specific order. I’d love to be able to pull out a Clerk-style nice literate explanation, but I’m not sure how to get there.
b. My current diagnosis of my struggles is that (A) there is a good solution to this, (B) I’m just currently not “holding Clerk right” yet.
I recall having heard @mkvlr comment on core.async as being opaque. In contrast, in Erlang, one can query the system for the current running processes. I feel like this is the problem I encountered with Playwright. The running system state was left invisible. I wasn’t sure when to def, when to defn, and when to (def ,,, (atom ,,,)). I felt like I ended up with a bit of a mess.
Two specific things I’m interested in:
• Whether I should be leaning into using Clerk when working with stateful systems at all, or just use the REPL.
◦ Since Clerk can be configured to run only when the user asks for Clerk to run (using the function nextjournal.clerk/show!), my hunch is:
▪︎ (A) yes, clerk is suited,
▪︎ (B) avoid using clerk/show! on namespaces that aren’t written to play nicely with Clerk’s caching behavior, and def-ed values could require resource cleanup.
• Thoughts on whether Clerk is a tool that can help make “external processes” visible, or if the caching & literal features make that hard.
◦ I realize that Smalltalk-like object browsers may perhaps be a better suited tool for this, as they (per my understanding) don’t usually rely on caching and immutability for their representation.
▪︎ Then again, I do generally prefer the immutable approach when one is possible.
◦ After all, Clerk’s tagline is Moldable Live Programming for Clojure, not Literate Programming For The Cases When Everything Is Immutable! (😸)
I now use a defonce for each mutable reference + a comment form with (.close mything) and (alter-var-root #' mything (constantly (start-thing))).
Really happy with the workflow. I can see what I am doing, and I can use Clerk to show the kinds of things I want to see.
A kicker was to embrace both REPL and Clerk, and keep “mutating commands” in a (comment ,,) to run only when I want to run them.
Hey @teodorlu thanks for sharing your experiences here - I've run into this stuff a lot when trying to create database-related examples 🙂
> (B) avoid using clerk/show! on namespaces that aren’t written to play nicely with Clerk’s caching behavior, and def-ed values could require resource cleanup.
Did you reach a conclusion on this point?
Not really, but that seems like a good place to start.
Presumably, Clerk doesn’t redefine defonce vars?
That way, I could safely use clerk/show! in my “explore mutable behavior” namespace, and then look at how I can structure the code for a nice interactive experience.
Thanks!
Note: I felt that what I wrote was a bit long, and perhaps “too big for slack”. I’m happy to move it off somewhere else if there’s a different place better suited to this.
Have you explored using defonce for the vars that are just handles to external mutable containers?
@taylor.jeremydavid Yes! I feel like I’ve found a workflow I’m happy with.
I either write a namespace to explore and learn, or to be used from other code.
To explore / learn, I now (defonce state (init-state)) with an (alter-var-root #'state (init-state)) below. That lets me quickly open the namespace in Clerk and it works (less friction than manual initialization), and also restart if possible.
I close and restart resources manually from the REPL. Since it’s a defonce, I can refer to the stateful resource easily.
Specific example from yesterday, using Playwright’s (stateful) API attached. Note: I haven’t cleaned up this code. It contains some messy exploration, but that’s kind of what you’re asking for.
I assume you’re thinking about XTDB examples? I’m interested in how you’re approaching that.
To answer more precisely what you’re asking for:
>> (B) avoid using clerk/show! on namespaces that aren’t written to play nicely with Clerk’s caching behavior, and def-ed values could require resource cleanup.
> Did you reach a conclusion on this point?
I now either:
1. Have an “explore namespace” with a toplevel defonce, only to be used in development (could be under a dev/ path active only under dev)
a. Example of “explore namespace” above ☝️
2. Or avoid toplevel def-ed resources completely.
a. This is how I prefer to organize “library code”.
My process is roughly:
1. Put everything in an “explore namespace”
2. Gradually move reusable pieces to “library code”.
Thank you again! > I assume you’re thinking about XTDB examples? Yes indeed, and I've tried a couple of times to figure out a more sensible workflow but have always given up prematurely due to time pressures to write the actual usage examples 😅 I will have to digest your write-up and report back soon 🙂