You can’t watch it from the client, so I presume the e/watch is server-side, and you’re reading the resulting value client-side. In this scenario, you’ll get the whole sequence every time it changes (modulo Electric relieving some of it). If it goes into for-by server-side before doing something client-side in the closure of it, then you’ll get diffing.

Yep your clarification is correct. Thanks that makes sense with the for-by. So if I did (e/server (e/for-by …)) on a sequence on the backend. Then when I add data to that sequence and remove data from it on the backend. The frontend will only ever receive the diff adding and removing to the rendered output? If so this may radically simplify the design I was considering.

Though I am not rendering to dom and do need the full sequence to pass to GPU at present so maybe it doesn’t quite do it.

You for-by identity-fn, which can be as simple as a keyword, if your sequence contains maps, for example. Electric will tear down and build up branches based on that identity function. For example, in this case the branch for 2 will be torn down (which means a small amount of communication with the client), and the branch for 4 will be mounted (which means an amount of data transferred depending on your e/server and e/client blocks inside SomeFn)

(for-by :id [m [{:id 1} {:id 2} {:id 3}]]
  (SomeFn. m))

;; =>

(for-by :id [m [{:id 1} {:id 3} {:id 4}]]
  (SomeFn. m))

Perhaps this is clearer:

(e/server
  (e/for-by :id [m [{:id 1} {:id 2} {:id 3}]]
    (let [id (get m :id)]
      (e/client
        (dom/div (dom/text id))))))

;; Initial payload to client: 1, 2, 3

(e/server
  (e/for-by :id [m [{:id 1} {:id 3} {:id 4}]]
    (let [id (get m :id)]
      (e/client
        (dom/div (dom/text id))))))

;; On update: <teardown instructions for 2>, 4

This is much clearer. Thank you.

I.e., only the value of :id passes the client/server border, so only that is transferred. No sequence, and no map.

Oh that is informative. I suspected that the whole map was transferred but it was just checking on that particular value. I would definitely need the entire map.

Then you just pass m across the border, provided it contains only serializable values.

You will then get whole map payloads for the value of m, if anything changes within.

I.e., if your map has 1000 key/value pairs, and one changes, you will be transferring 1000 key/value pairs. As opposed to when destructured on the backend. This is an extreme example, it won’t be a problem most likely. Just highlighting the logic of what a unit of change is to Electric in this case.

My current thinking is I keep a count of the sequence and perhaps a hash. Then I could use e/for-by and accumulate it into a set on the frontend and do a count/hash check before passing all the accumulated data into rendering. But I’m unsure if this is the best way to go instead of just doing a more manual diffing process. Like batching the fetches and removing data I don’t want in the sequence anymore on the frontend

My maps are small. But the sequence is potentially large.

My advice would be to not worry about the payload size initially. Go with as plain logic, and basic Electric code as you can. Trim it later when it actually works, and the logic has solidified by moving client/server borders.

But definitely use for-by if you can.

I haven’t worked on a solution yet as I was planning to use tmdjs but it was causing fatal errors with missionary

This is what I have at the moment. It's sending the entire sequence every change on demand with no buffering of data. So I would like to optimise this.

realise I could have shared this earlier as some grounding to the context. But yea I will go with for-by and keep it simple.

I also was not so worried about payload size and more thinking about network request reduction. Like sending and receiving every data point might be silly rather than getting the next chunk at a certain point along the scrolling and just batching the request so plenty of data is there ahead of time.

I believe leo did some work on this here but it hasn’t been integrated into electric yet https://github.com/hyperfiddle/electric/blob/46f9a0dc39f10afcb1fb80d6ff58c50f2ea04055/src/hyperfiddle/incseq.cljc

interesting

Oh, and the stricter requirements (and improved warnings) of IC has revealed a handful of things in our code that were just plain wrong, but kind of slipped through the cracks in our current code. Really nice improvements on communicating problems to the developer.

What is ic-squashed?

https://github.com/hyperfiddle/electric/commits/ic-squashed

Electric with incremental compilation: “IC”

I have not yet deeply reviewed the IC changelog, one thing I note as missing is that reader conditional behavior i.e. #?(:clj ... :cljs ...) is completely changed

i think it now does "the expected, least surprising" thing, which is to say things like (e/client #?(:cljs ...)) can now properly guard cljs-only stuff

in Electric v2 reader conditonals would evaluate on JVM always i think, making them basically unusable

I might have some details wrong, I am not really paying close attention to this workstream yet

Ah, gotcha. I assumed that “hiding” branch like that would lead to sync problems between client and server.

another example: (e/client #?(:cljs #js {"a" 1})) will do the obvious right thing

I am not completely sure about the "hiding" issue under IC, but hiding a client-only literal i think is harmless and works

One place I had to do this was in the uix macro we use to mount React components (it would complain that uix.dom is an unknown class). It probably would have worked OK if I wrapped the two branches locally. I ended up proxying the functions and made them no-op server-side.

Are you pulling the branch regularly? There was e.g. a fix on requires yesterday. If you provide repros of the issues I might take a look at them. Order of preference is • rcf test • minimal repro in starter app with complete reproduction steps

Hiding a branch with a transfer will produce a runtime crash

Yes, running on latest. Alright, I will try to capture as I proceed.

1. you have incredible patience to tolerate 45 sec compile times, I moved to ic after like 12 2. what's the exact error when this happens? 3. does dom/style work? 4. just checking, did you add the shadow build hook? 5. I think this is actually a feature

1. It’ll be great for sure, that’s a big reason why I want to be as prepared as possible. At the same time, it’s WIP and might change at any moment, so sticking with the current version is better. I’d be prepared to say that even with 45 second compile times and occasionally surprising behaviour, Electric is still better than the alternative. 2. There’s no error, it just silently doesn’t work. 3. Actually, I looked this up, I see that it’s the same problem that was fixed in main. It’ll be fixed when ic-squashed is rebased on master. 4. Yup, it’s been added. 5. I guess it depends. Simple values are easy to wrap in conditionals. More complicated things might require more sophisticated solutions, I guess. Personally, I was just caught off guard, since it wasn’t in the (WIP) changelog.

@U09FL65DK Is there a particular category of bad code I should be looking for when "Index 2 out of bounds for length 0"-style errors occur?

cljs code should work in an e/client block, the exception I know for now is #js literals. I mean they work on cljs side but the clj reader doesn't have a tag reader for it.

index out of bounds runtime error is usually a symptom of bad macroexpansion, i.e. the server expanded the code differently than the client

Ah, so I could be looking for CLJC files required in a #?@(:cljs … block etc. Thanks.

I’ve observed the CLJS in e/client block failing with cljs-bean. I’ll see if I can get a clean repro for you.

There’s no error, it just silently doesn’t work.That's probably my biggest issue with electric so far in daily practice, including its appetite for eating many/most exceptions. So I'm excited that you're focusing on this (more/better error msgs). Dev happiness ftw.