absolutely possible, sky is the limit to introspect and reflect the DAG and in depth observability

Here is the Electric "DAG bytecode" – https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_compiler_internals.cljc

I think it would be straightforward to visualize that using GraphViz or whatever

That would be step 1; step 2 would be to observe the events on the wire protocol in order to animate the graph and see values flow through it

Separate idea – a "reactive REPL"

in a simple terminal repl, you type a form and press enter and get a result. Request/response

Imagine that same terminal UI except in a browser and the response is animated like the Two Clocks demo

user=> (println (e/server e/system-time-ms)) 1692449153122 ; animated

each REPL interaction is a live app and stays live until you delete it

the blocker to this is that Electric is still a compiler (no eval) so you have to wait on shadow rebuilds

in 2024 Electric will become a JIT (like the JVM and also Clojure), moving the compiler analysis to runtime, which will enable such a REPL without iframing a second shadow instance or something

the live-coding demo video on our homepage http://www.hyperfiddle.net uses a second shadow instance with an iframe (and the video is also sped up to hide the hot code reloading pauses)

A cheaper alternative is making Electric compatible with SCI, which has no blockers but someone has to do the work it's probably a few weeks of work that we dont have time for, perhaps someone could get Clojurists Together funding for this

Finally we still are looking for notebook infrastructure a la NextJournal that is unbundled and separate from any application level concerns. "Session-lived backend" is the term for an on-demand "flyweight" backend to spin up and host a throwaway JVM and classpath for livecoding "fiddle" infrastructure. This missing piece gets end-user programmers the ability to build real apps (that have dependencies and need classpath control, i.e. can safely connect to some arbitrary database from userland)

we have only run it so far in onprem, there are some threads discussing the websocket issue if you search this channel and #C03RZMDSH

👍 I searched for Ions and only found a brief mention from 2018; will look more for websockets 👀

search by @U0514DPR7 they are his threads

Thank you 🙂

Seems like it's probably possible through API Gateway's websocket support. Just need to abstract the session management outside of the electric adapter. Is there any documentation on different adapters? Seems like I could reuse most of the logic from hyperfiddle.electric-jetty-adapter?

cross linking other thread https://clojurians.slack.com/archives/C03RZMDSH/p1692310904023429

https://clojurians.slack.com/archives/CL85MBPEF/p1676057124117659?thread_ts=1675723956.514279&cid=CL85MBPEF.

@U5JUDH2UE if you would summarize the issue please in as few bullet points as possible, that would be helpful

• Electric requires the WS session to be handled by the same server instance for the duration of a user's session due to its inherent stateful nature. (Running the server side version of components and sending the client any data when the client expects to receive it.) • Diatomic Ions are useful for deploying Clojure functions in a scalable way while also having very fast Diatomic DB access. • Ions can be exposed either through a Lamda proxy or API Gateway HTTP endpoint. • There's no way to route either to always be handled by the same server instance when scaled horizontally. • This could be handled by placing WS events in something like Redis and relying on each instance to publish messages it received and subscribe to messages related to sessions it's currently serving. • Doing so would also require a custom Electric adapter which abstracts the API Gateway WS abstraction where new events being received are translated to HTTP requests and sending events requires calling an HTTP endpoint. • Ions also don't seem to provide any lifecycle hooks when being scaled down, which would be a major problem for long-lived Electric sessions which would be lost and require a user to refresh/connect elsewhere. All together, I'm not sure Ions are the right deployment solution given Electric's design. The server/client needing to run the same code in parallel and communicating throughout is, I think, inherently different from what Ions expect and likely requires a more traditional server deployment.

That's not very few 😬 More of the story is that it probably shouldn't be done, I think.

Thanks, I'd guess the path forward is to reject that sticky session routing to an ion is impossible and figure out how to do it

anyway isn't this a dead product? Nubank uses Datomic Onprem

I think you might be right about that. I'd never looked closely, but I was getting a sense that it may not be desirable to use even outside of this limitation.

Dustin… you’re… saying it… out loud… I’ve been “regularly” asking them if they’ve started to use Datomic Cloud at Nubank. Never got a response. So: they didn’t yet start to use it. Still, the way they develop first for On-Prem, but in a Cloud-compatible way, then release things for Datomic Cloud a few months later, proves that the product is not yet dead at all in their mind. I don’t know if it’s just to support their existing Cloud clients, or if they’re really investing in it for the long run. I’ve got the feeling they want the long run for it, but might someday unify things so much so that the CloudFormation templates for configuring AWS would also become useful for On-Prem (Clouded On-Prem).