Fork me on GitHub
#matrix
<
2024-05-22
>
kennytilton12:05:03

Someone is thinking about using MQTT to have MX, starting on Flutter/MX, work across independent processes. Thoughts? 🙀

kennytilton13:05:37

One primordial question: do we have a separate structure/software, call it MQX, that 1. sees only other MQXs, but which works reactively with existing MXs, or 2. just teach the existing MX about MQTT and the existence of remote other MXs? I should add an even more fundamental idea, which I am taking as given, is that only one MQX controls the MX datapulse lying at the heart of state integrity. Not sure that is necessary, so room for meditation there as well.

chromalchemy21:05:25

I dont know a lot about it but this project uses some sort of heartbeat pulse to synchronize state in “virtual world”s. Might be some food for thought. He’s implementing in Elecric, but the project/concept has a longer history, relating to crdt’s and such. https://github.com/NikolaySuslov/krestianstvo-electric

đź‘€ 1
kennytilton09:05:51

Thx, I posted in part to see if anyone knew of prior art. I did know about Electric, but not the heartbeat pulse. I learned from Pythonista Philip Eby, who started from Cells to create the Trellis reactive Python lib, about David Gelernter's Trellis in which the universe is a cellular automaton that moves in lockstep with a single pulse. That would be simplest, but some morning soon I will load up on caffeine and stare at MX with the idea of making it work with multiple pulses. It might just come down to the internals recording a GUID pulse-ID with each pulse...ah, but who is in charge of the recursive state propagation step?! That's where the caffeine will come in. 🙂

chromalchemy13:05:36

It’s all over my head, but that’s where this Croquet stuff caught my eyes.. With peer clients, who’s in charge? leads to.. recursive “universal” objects lazily sampling time from each other? 🤯

kennytilton15:05:19

Heh-heh, I will get back to you after my exploration. In part, MX should be OK because part of the "am I current?" calculation is asking all existing dependencies if they are current. Likewise, new dependencies read for the first time during an evaluation ask them JIT if they are current. MX does not yet allow cycles, so eventually everyone decides they are current and the evaluation completes. Now it gets tricky. In solo MX, everyone records the current pulse so we can do a fast determination that they are current. But which pulse do we record, if every process has its own pulse, and formulas are sampling from multiples? With a single pulse keeper we have only one pulse, so no problem. But then almost no state gets to use this shortcut, because every change in every client effectively invalidates the entire multi-client cache! Hmmm, but even then it just means that the MX internals run around and mostly decide they never found an uncurrent dependency, then flag themselves as current without even running. Note to self: do some heavy instrumentation before attempting this. 🙂

chromalchemy21:05:19

> radical synchronization system with the notion of virtual time

chromalchemy21:05:47

But maybe thats only for clients?

kennytilton13:05:09

Well, that is the win: we only need clients! I was def thinking client-only, but even then there is a server-like option: the first client of any given app becomes Keeper of the Pulse, so there is only ever one pulse source of truth. So maybe a performance bottleneck. Unless we let each client also track its own pulse, and let state propagation proceed if no outside client state will be affected. There might still be a performance hit, but perhaps less so. And it comes down to cases. It always comes down to cases! If we are just looking for a way to make normal MQTT traffic look like MX, hiding the necessary MQTT wiring, then the volume would be no different than any MQTT app. In this case, we create an MQX (Matrix MQTT) that has its own little API and is used only for CSP traffic. MQX then is just a friendly wrapper for MQTT. We would still have MX properties reacting to MQX changes, and MX watches could trigger MQX changes. Not sure MQX props can watch MX props, tho. Perhaps at that point we may as well forget MQX and just teach MX about MQTT. :thinking_face: Might be better to pseuso-code MQTT into MX and see what happens. I did this once when I had to because the dev demanded a spec, went really well because it forced me to get specific. I should do that on the f/mx wiki so it creates a record and reference, too, if all goes well.

kennytilton13:05:50

A pure client-only solution would simply allow multiple KoPs (pulse keepers) and convert the current integer pulse to a KoP-integer pair. But MX also has a "finish business" fin-biz function that sees to the thorough and non-redundant propagation of a single change. I guess we could still do that, as long as we had a way to single-thread the clients' fin-bizzes. Finally, the devil on my shoulder is pointing out that MX went quite far before fixing the glitch thing. Sure, things got computed twice in one pulse, with different results, but the chaos converged without triggering app misbehavior. Until a rare case where it did not. But maybe that had a different solution than single-threading change? Fun stuff.