Fork me on GitHub
#hyperfiddle
<
2023-05-08
>
Vincent00:05:46

Wow super cool! @tlonist.sang i'm really glad it's up so i can read through the source and learn more. Looks neat, is it for "confirming" attendance to events?

tlonist00:05:43

yes it is. What it does is • user enters and sees the marks on calendar on the days he/she attended • user marks today as attended • all these with authenticating tokens

Nikolai04:05:18

Want to share the prototype of implementing the Croquet VM in Electric - Clojure. https://github.com/NikolaySuslov/krestianstvo-electric In Croquet application architecture the state is never going outside the client and is updating in sync by using local internal queues of messages (Future message sends). These queues advanced in virtual time by receiving the control tick (metronome) message from the Reflector. All user-land events are also stamped by Reflector and distributed to all the clients back (sending owner too), forming very low net traffic. I am interested in how this suits Electric, and if this type of architecture can be realised even without introducing Croquet on top of it?

🙌 11
❤️ 7
2
Dustin Getz12:05:35

This is super cool, and impressive! Extracting some links while I read it: https://www.krestianstvo.org/ https://docs.krestianstvo.org/en/introduction/ https://en.wikipedia.org/wiki/Croquet_Project https://www.krestianstvo.org/docs/about/publications/ https://blog.codefrau.net/2021/08/what-is-croquet-anyways.html

; Reference implementations:
; Virtual Time 
; Reflector server 
; Future Message Send  ()
; TODO: Portals 

Dustin Getz12:05:33

> I am interested in how this suits Electric, and if this type of architecture can be realised even without introducing Croquet on top of it • Electric runtime is backpressured, which means that if you pause the network (try it with browser dev tools), messages are queued and will go through when network is unpaused. We see "offline as just online with extreme latency". This builtin capability seems to overlap a lot with this userland event system • The mouse cursor is lagged presumably because it is round tripping through the server? We're working on improving our optimistic state abstractions for cases like this (we are focused on perfecting our crud forms first, but the underlying primitives should be the same for things like mouse cursors)

👏 3
Dustin Getz12:05:22

I don't really understand the Croquet architecture yet, specifically I don't understand this exactly yet: > In Croquet application architecture the state is never going outside the client and is updating in sync by using local internal queues of messages (Future message sends). These queues advanced in virtual time by receiving the control tick (metronome) message from the Reflector. All user-land events are also stamped by Reflector and distributed to all the clients back (sending owner too), forming very low net traffic.

Nikolai12:05:10

Yes, there is a good explanation of Croquet VM on their current version (Croquet OS) https://croquet.io/croquet-os/

👀 2
Nikolai12:05:02

Croquet OS is closed sourced, but it has the same architectural components, that I try to implement currently (based on Virtual World Framework aka Croquet IV)

Dustin Getz12:05:20

What is your relation to this project? Krevistiansvo is your project?

Dustin Getz12:05:16

"Croquet OS provides a kernel that enables bit identical shared simulations between users." Is there a rationale for this capability? Why is bit-identical simulation something that virtual worlds need

Nikolai12:05:00

Yes, I am a founder and developer of Krestianstvo project. I am working with Croquet since Smalltalk version.

Nikolai12:05:56

Sure, bit-identical simulation is a crucial concept for virtual worlds, as when simulation is going no network traffic is involved. Clients getting only timestamps from reflector, which advanced the simulation.

Dustin Getz12:05:29

"when simulation is going no network traffic is involved" -> what about syncing user I/O?

Nikolai12:05:24

All user-land events needs to go to Reflector to be stamped and then distributed to all the clients back. That's considered as the minimal traffic (like mouse coordinates or clicks). The application state is stying on client side.

Dustin Getz13:05:49

how is latency mitigated?

Dustin Getz13:05:19

Ah, I guess the simulation state excludes cameras for example, it's the scene graph which is synced and consistent?

Nikolai13:05:19

Right, users operate through local Views on a synced Model.

Dustin Getz13:05:25

if the model state is a reduction over global event log, and this reduction is replicated on each client (peer?), then the global event log is replicated to each client/peer, so sorta like a bitcoin node? What if the number of events is large

Nikolai13:05:48

Yes, that's great introduction! As for the latency, clients with low speed network will not pause the clients with hight speed traffic, that's was the key feature of Croquet. In the contrary to lockstep algorithms.

Dustin Getz13:05:50

So the reflector runs in the edge, all events are funneled through the reflector (which is like a transactor, it decides a globally consistent total ordering). It sounds like https://www.cloudflare.com/products/durable-objects/

Dustin Getz13:05:16

To map to electric/missionary: synced Model - this is discrete event streams, in simplest webapp terms this is basically the database transaction log? local Views - this is continuous time lazy rendering, electric maps best to this part

Dustin Getz13:05:22

The question of if the synced model (the transaction log) is replicated or centralized - Electric doesn't care, we would say this is a data plane decision, some databases replicate and others are centralized

Dustin Getz13:05:04

The Croquet event layer is probably better than ours, this is the part where there is overlap, it is comparable to our optimistic crud form sync abstractions which we haven't really talked about or shown yet. Our optimistic form state sync is done in Electric userland not the runtime

Dustin Getz13:05:21

So it makes sense that there is overlap, and your project makes sense in an Electric worldview

Dustin Getz13:05:42

Did I get this right?

💯 2
Dustin Getz13:05:46

Have i answered your original q?

Nikolai13:05:43

Yes, thank you Dustin! I am really excited with Electric and would like to do a lot with it!

🙂 2
Nikolai13:05:53

That's demo video of the prototype.

Dustin Getz14:05:06

Is this accurate: The big idea is to use the Electric distributed runtime as a basis for a higher level scene graph sync abstraction for use in collaborative virtual worlds.

💯 2
Nikolai01:05:43

Here is a good explanation video of David P. Reed at OOPSLA '05 talking about Croquet architecture. "Designing croquet's TeaTime: a real-time, temporal environment for active object cooperation" https://dl.acm.org/doi/10.1145/1094855.1094861

👀 2
Nikolai14:05:33

Here are the slides from the presentation.

👀 2
Aziz Aldawood11:05:03

@dustingetz what are your thoughts on next.js server actions?

Dustin Getz12:05:19

I think (last i looked) server actions are RPC (meaning not a scalable composition model), tbh I struggle to manifest the attention to slog through all the complexity and adhoc machinery to figure out all the ways their thing is broken, perhaps someone who has used it can compare

braai engineer16:05:49

Anyone need help at their company porting any apps to Electric for the performance benefits? I am looking for project work and would love to spend more time working with Electric .

🔆 4
2
3
Erich Ocean21:05:45

Has anyone integrated a standard 3rd party React component with Electric? I'm looking for sample code.

Erich Ocean21:05:51

Yup, that's enough. Thanks!

Erich Ocean23:05:40

It seems like new is being called on the result of the e/fn call (macro?) here: https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#L38-L41

Erich Ocean23:05:44

Specifically, why not just (F v) on line 33?

noonian23:05:22

new is used to call electric functions. Since F and InputSubmit are electric functions you have to call them with new. Note that the e/defn macro will rewrite the code so it has the semantics of a function call and is not a regular usage of new

Erich Ocean00:05:09

Got it, thanks.

Vincent23:05:13

other than e/for-by are there any other constructs that iterate over collections? I have a simple query return result sets and i want to draw them, but not sure how to e/for-by it.

Vincent00:05:40

I get a #{set} of [vecs of {maps}] so it's interesting figuring out how to unpack it

Vincent00:05:14

#{[{:details "deets'}] [{:details "detes"}]} not sure how to `e/for-by it

Vincent00:05:37

ideally the input to e/for-by is a vec of maps?

Vincent00:05:51

I got it working but it looks mental 😅

tracks (-> (xt/q db '{:find [(pull ?track [:track/id :track/album-id :track/title])]
                               :where [[?track :track/album-id album-id]]
                               :in [album-id]}
                             album-id)
                             vec
                             flatten
                             vec)]
vec flatten vec :face palm:

Vincent00:05:33

and then for whatever reason calling it just track works instead of track/title

(dom/div (dom/props {:class "track-cool"})
            (e/server 
              (e/for-by :track/id [{:keys [track/id track/album-id track/title]} tracks]
                (e/client 
                  (dom/div (dom/text (str "track " title)))))))
i think that's by design?

braai engineer08:05:27

@U055PQH9R4M the flatten will be expensive. You can do (map first (x/q db …)). @dustingetz is it advisable to pass a sorted seq to e/for-by or does the keyfn imply a sorting seq? I assume you’d want a sorted result but also keyed for efficient updates.

Dustin Getz10:05:22

e/for-by does not sort, it is a sequential traversal same as clojure for

Dustin Getz10:05:16

e/for-by performa diffing internally for stabilized reactive updates, which is a traversal, so pass reasonable collections (like a page of database records, not 1000 records)

Dustin Getz10:05:01

the DOM itself can only handle about 2k nodes anyway

braai engineer11:05:50

@U055PQH9R4M I think you’ll want to add a :sort-by clause to your XT query, or sort-by the query result before passing to e/for-by to avoid the diffing, or does the diffing always occur?

xificurC12:05:02

diffing is necessary to handle updates effectively. It's the same as in e.g. react where you have a dynamically updating collection and need to provide a key

Dustin Getz13:05:48

to be clear, diffing is separate from sorting. e/for-by renders dom elements in order, it can render any sequence not just UIs that are in sorted order