Clojurians Log v2
biff 2026-11-12
Trying to get my head around how the SSE paradigm works. I notice that biffweb.datastar.ui/app-page produces a “text/html” response map in both SSE and non-SSE paths. Presumably in the SSE path, that “text/html” response isn’t used directly - only the :body value is extracted and sent over SSE connection? Am I on the right track?
Also, the datastar-clojure SDK example uses a “connections” atom to loop over the SSE connections. My reading is that biffweb.datastar doesn’t use that approach? I couldn’t find a similar atom. Looking at streaming-response-body, it seems to run the handler in a loop every 20ms. Is wait-with-heartbeats / wait-for-refresh what pauses that loop between refreshes? What does the lock do? Just interested in how it fits together
> Presumably in the SSE path, that “text/html” response isn’t used directly - only the :body value is extracted and sent over SSE connection?
yep--`streaming-response-body` ensures the status is 200, then extracts :body: https://github.com/jacobobryant/biff/blob/bb789b6bdd2008767f6649865c7e8174e1832d5e/libs/datastar/src/com/biffweb/datastar/impl.clj#L296
> it seems to run the handler in a loop every 20ms.
at most every 20ms to be precise.
The approach biff takes is that there’s a single Condition object stored in the system map (see https://github.com/jacobobryant/biff/blob/bb789b6bdd2008767f6649865c7e8174e1832d5e/libs/datastar/src/com/biffweb/datastar/impl.clj#L218) instead of storing each connection. You can have a bunch of threads that all sleep until the condition is signaled (https://github.com/jacobobryant/biff/blob/bb789b6bdd2008767f6649865c7e8174e1832d5e/libs/datastar/src/com/biffweb/datastar/impl.clj#L249). So when a db transaction is committed, we signal the condition, and then all the threads wake up. If it hasn’t been at least 20ms since a thread last pushed some html to the client, then it’ll sleep until it has been 20ms.
> Is wait-with-heartbeats / wait-for-refresh what pauses that loop between refreshes?
yep 👍 and the heartbeat thing is there because if a client disconnects (e.g. closes the browser tab), we don’t find out until we try to send an event. So the heartbeat will potentially throw an IOException which streaming-response-body catches.
> What does the lock do?
The Condition and ReentrantLock things go together. there’s some non-obvious stuff that goes on between them, mainly that when you call .await on the condition it will release the lock, and .await won’t return until it re-obtains the lock. so even though you can have a bunch of threads all sleeping inside wait-for-refresh at the same time, only one thread in the process can hold the lock at any given time.
Basically it’s just java pattern for notifying a bunch of threads when something changes (the :biff.datastar/epoch atom in this case).
Thanks very much for the explainer @foo!
So the original SSE request map is stored as a closure inside streaming-response-body and replayed over and over to the handler right?
no problem!
> So the original SSE request map is stored as a closure inside streaming-response-body and replayed over and over to the handler right?
yep 👍
How do you find using the database for tab state for a small-medium sized app with multiple pages/endpoints? I’m thinking of storing things like whether a sidebar is open, but wonder I’m going to end up with an explosion of tab-state columns for relatively insignificant things. Do you end up with a lot of detail stored in tab state to account for the things that would previously have been transient client state? I wonder how suitable a relational table ends up being, and whether a nested data structure in memory might be more flexible, especially if sharing tab state storage between multiple pages in the app.
I do end up with a lot of things in tab state. FYI although I store it in the database, I’m storing tab state as unstructured data in a single column (EDN serialized with nippy, stored in a BLOB column), not defining sqlite schema for everything in the tab state. https://github.com/jacobobryant/biff-starter/blob/306292a4a817f956106defbd52741abcd55cab9e/src/com/example/schema.clj#L10. and the malli schema (`tab-state-schema`) is used to validate before writing.
Also, although I haven’t actually done this with my apps yet, I think it would make sense to structure the tab state by page, like {:page-1 {…}, :page-2 {…}}.