Clojurians Log v2

Wow thank you for these updates to statecharts, my system is fully using all of it to VERY good effect. The git backed persistence is allowing my agents to recover across sessions beautifully.

Nice! glad to hear you’re seeing similar benefits

I incorporated these changes into my current project and they are awesome! Thank you very much for these changes. I did notice a few things. The digest function is awesome for letting the LLM see the "flow" of the app. The only problem is it shows too much data by default. I have a web based chat ui for AI agents, and just a short conversation with tool calls was enough data to fill 200k context from the AI calling rt/digest. I had the AI create a ui_digest tool that offers progressive communications, showing the top level, and truncating long strings and values, with the option to dig deeper as needed and that is working well. I wrote my entire AI agent framework in statecharts, and am using them from within fulcro by starting a simple env with event loop. I have mutations setup that allow fulcro and the statecharts to communicate. The new headless setup is much better for this, but is still not perfect. I would like to be able to have this coupled better, but I am not sure how to make it work. I would love to have statecharts that run in clojure with full access to fulcro, without having to run 2 different environments with mutations bridging them. I feel there is a ton of fulcro and fulcro-rad functionality that I could use from my statecharts if this was more tightly integrated. I'm not a programmer by trade, just a hobbyist so I am not sure how to make this work better. I really like statecharts for full control and introspection, and have been using them for a couple years now for most of my projects. They are especially useful to AI because the AI can see everything easily with almost no special tooling required once you create a good concise prompt on the tools available in statecharts, like the simple and runtime namespaces. I like the direction you are going and look forward to seeing the statecharts changes you are hinting at here.

We’re launching a new project and are in the midst of writing a statecharts-integrated version of the RAD primitives…not sure if/when we’ll release that. Not sure what you mean with the digest bit polluting your window. I’m not using that directly in my AI flows much. In terms of comms I’m also not sure 2 diff envs? The RAD stuff is already able to be used from statecharts…you just make a new state type that starts the UISM. I’m working on ui_routes stuff in statecharts right now (about to release a big bump in that) and when done the whole ecosystem will be aimed at composable systems as statecharts.

Our results with headless testing dev with LLMs is spectacular…Super fast. One-shot getting things completely right.

Fulcro can start a statechart as part of the client db env in clojurescript, but I want my agent loop to run whether the browser is running or not, the web ui is just a way to see what is happening with the agents. Currently i could not figure out a way to get fulcro to start these statecharts integrated, so I have a bootstrap that runs from server.clj that starts a simple env and event loop. Then I created mutations, for example the web ui chat uses a mutation to send the conversation to the agent statechart, which processes it, and sends it back using a server push over the websocket remote so the response shows up in the chat ui.

Are you using headless CLJC stuff? Sounds like “no”…for websockets your setup would be a little more involved, but I think still possible. The latest Fulcro even has a CLJ implementation of a lot of React Hooks (though I prefer not to use those for real logic/data needs). I’ve built a “headless” skill for AI, and a CLJ stubs skill (which explains how to write a “working but minimal” CLJ version of things). For example I tell it for UI components (like js dropdowns) just make a wrapper that converts it to a dom select with the proper callbacks made available. For things like pretty graphs just emit a div and convert all attributes to have :data- prefixes so the tests can see the inputs sent to the component

I converted it to use the headless stuff from 3.9.3 last night. It seems to work well, but I still am using cljs for the web ui with statechart routing. I don't want a fully headless setup, more of a hybrid one. The web ui is great the way it is, I just hate how I have to bridge to the statecharts that are running outside fulcro using mutations. To me headless implies no browser, but I use the browser for a chat interface, and various rad reports that pull data from the agent statecharts using mutations.

The agent statecharts run contantly, so I don't want that to stop if I close the browser. I originally was using statecharts started in cljs but when I'd close the browser it would all stop working. That lead to this hybrid setup where I have a statecharts simple env started from server.clj and the mutations for communication.

Here is a link to an older app where i started playing with this setup, in this app I kept it all to 1 component. https://gitlab.com/michaelwhitford/gailish/-/blob/main/src/main/us/whitford/gailish/components/mattermost_bot.clj?ref_type=heads#L633

In my newest app that is unreleased yet I am starting an nrepl server, and the statecharts simple env from server.clj so that the AI can access the entire system from the nrepl. For dev I have it start the shadow-cljs repl, and with my clojure-mcp-light tools it can access both nrepls as needed.

I’m using the term “headless” to mean you can run the UI and server in the server during LLM and tests…you still have the web ui as what users would use. The point is to have fast LLM interaction/turnarounds without needing a browser in that loop. On your statecharts issue: You’re saying that you want persistent statecharts in the browser…well, you could just make them fully server-side and just bridge it with what looks like an event queue (e.g. you send events to the server as statechart events, and it returns the new config). I’ve toyed with more advanced ideas (full-stack statechart where the chart runs in both, but that gets kinda crazy). You could also run the statechart with a new implementation of the statechart working memory store using browser local storage…lots of options.

I’m about to release statecharts 1.3.1 (1.3.1-SNAPSHOT is available now). I’ve been calling the ui routing “alpha”…the new version has a few minor breaking changes. Be interested in your feedback. The Guide.adoc was ai-updated on the routing-rewrite branch in github.

https://github.com/fulcrologic/statecharts/blob/routing-rewrite/Guide.adoc

That bridging part is where I use mutations currently. I click send on a chat in the browser, it fires a mutation that sends the data to the statechart as an event with a data payload. The statechart processes it through an LLM api, then when the response comes back it uses a push from fulcro to send that response back to the webui over a websockets remote.

The new routing stuff allows for full composition (e.g. an invocation state can have subroutes, and even be dynamically loaded module). The URL integration is better…and I think the params stuff…I’m still testing it before I actually release it to see how well it works headless and in UI under various scenarios…there is a demo in https://github.com/fulcrologic/statecharts/tree/routing-rewrite/src/routing-demo2/com/fulcrologic/statecharts/routing_demo2

Main API change is :route/path changed to :route/segment…the params are no longer part of the path ever…and even that isn’t required…defaults to just using the component’s name. If you see holes, let me know

I will have time tomorrow to explore these new changes, thank you.

sure….I’m trying them now and there is some breakage…had AI do some of the URL sync work, and it appears flaky at best…oh, looks like it was the demo that was the problem, not the code of the lib.

I decided to revert the original ns and put the new stuff in a new pkg…so I won’t break your code with the update…it was marked alpha, but I still think now that I’ve got more files it just makes it ok to deprecate the old thing instead.

I think I can live with that, I just incorporated 1.3.0 into my new app, and the old app can stay pointed to the earlier versions if you just want to go for it.

Turns out about 80% of what I had you eliminated with the async statechart support.

Yeah, that was a mind-blowingly good idea…honestly, my “laziness”/“lack of time” had kind of blocked it from my mind by not having time to write it…then I realized I could have AI give it a shot. Last friday I was literally starting a convo with AI and was about to type “help me figure out how to eliminate this cruft in js because of the stupid async nature of js…” and it hit me that all I wanted was blocking, which could be implemented as parking…it was a very meticulous and messy thing to do, but I knew AI was good at that kind of thing when it had a good ref implementation. It made a few mistakes, and there may still be bugs, but so far it looks pretty solid, and cost me very little time to do as a result.

I heard a comment the other day (a compliment to the clj-statecharts lib) that was “and you didn’t over-engineer it”, which I suspect was a “dig” at my version…but my version supports invocations in an completely arbitrary and composable way, deep history nodes (part of the spec), is pluggable in ways that allowed this easy evolution in many different ways (all of which I’ve used)…I can use it for everything I need…e.g. I use it for long-running CLJ charts with a transaction aware CLJ memory store/event queue (implemented in SQL)…Personally, I’m quite happy with the exact level of engineering I did. 😄 My teams even use it for things like server side job tracking of SQS submitting background jobs with UI progress tracking. We have a cluster running 50k+ persisted long-running (some for years) statecharts in parallel right this minute. I use it to as a scheduling system for nightly infra tasks within the cluster where one and only one node should run a job (my event queue guarantees exactly once delivery semantics on a chart, cluster wide…though the chart logic internally does have to be idempotent 😄).

I have written a fulcro-rad-git adapter to use git repos as memory, and feed forward communication mechanism for AI. I also wrote a git-embed rust util that turns a git repo of text files into a vector database with similarity search. git as history graph, knowledge graph, agent memory and communication, and similarity search in one. I feel I am doing very well with Opus and Sonnet, and a bit of local qwen3 thrown in.

nice

Your description of some of your charts, could you maybe ask your AI to extract the patterns into example statecharts in the repo? Those sound like great use cases for an agent swarm, but I'd want to use git as the backing...

The charts themselves are less interesting that the plugins: SQLDurableEventQueue, SQLMemoryStore, the thread pool/infra setup for running thread pools for event delivery across the cluster, etc. I’m guessing an LLM could roughly reproduce them from the ideas + the statecharts repo source. I don’t really have the time to do an extraction/verification/etc…even driving an LLM…too many things to proofread already,

I pasted your last comment and Opus is starting to design a git backed persistant statechart. hah.

exactly

it’s all an idea and a bit of nudging away

statecharts 1.4.0-RC1 released

FYI, I just realized I have a poor choice of signatures onthe routing busy hook,..I’m doing tons of review. The 1.4 API should be considered slightly unstable until I’ve worked on it a bit more. The problem with busy was that it was giving the ROUTE statechart info, and you more care about the local component. So, the signature will change to accept [app ui-props] which is much easier to use.

It's no problem for me, I have a full set of documentation and an agent that I can spawn to update it when you change it upstream, and then schedule more agents to integrate. git-backed persistence is pretty amazing.