i don’t understand the question?

e/for-by is under server, but couldn't the whole thing be done on the client? Something like this:

(e/defn Chat []
  (e/client
    (try
      (dom/ul
        (e/for-by identity [msg (reverse msgs)] ; chat renders bottom up
                  (dom/li (dom/style {:visibility (if (nil? msg)
                                                    "hidden" "visible")})
                          (dom/text msg))))

      (dom/input
        (dom/props {:placeholder "Type a message"})
        (dom/on "keydown" (e/fn [e]
                            (when (= "Enter" (.-key e))
                              (when-some [v (empty->nil (.. e -target -value))]
                                (e/server (swap! !msgs #(cons v (take 9 %))))
                                (set! (.-value dom/node) ""))))))
      (catch Pending e
        (dom/style {:background-color "yellow"})))))

This is a common pattern. Since msgs is a server value you want to diff it on the server. e/for-by diffs and maintains the list incrementally. Doing the whole thing on the client will compile but you'll be re-sending the whole list of msgs on every change

I see. And will all the li elements be updated because their value changed, or will it only add the new msg and shift the others positions in the list?

good question. Updates are fine grained, so if a new value comes in it's the only one that's going to create

So the sever will only send the info for the new msg and the old one that was discarded?

correct

Impressive!

To mount A xor B you need a conditional, e.g.

(if-some [txt (try (e/server (get-text)) (catch Pending _))]
  (dom/div ...)
  (Spinner.))

(let [!mounting (atom true)]
  ...)

@U6D1FU8F2 i recommend not trying to battle with Pending if possible, just let the page stream in for now

I say this because there are some aspects of Pending that we are internally a bit confused about (e.g. does Pending always match what you'd want to be "Loading" or is Pending sometimes seen at times that you would not consider to be Loading – we aren't quite sure)

If you need to do this it's fine, just stay in touch so we can understand your use case

I’ve run into cases where I’ve found it a bit hard to get things to happen in a desired order wrt rendering. For example, in a list, I’ve found that it’s hard to fix the race condition of items being added to the top of the list vs. scrolling to the currently highlighted item. If the added items pushes the highlighted item below the fold, scrolling to the highlighted item will pretty much never happen, since the data that triggers the scrolling event (i.e.: the list content changed) is available before the DOM elements are mounted on screen, rather than happening in the “same transaction” so to speak. With Pending, it seems that Pending -> Data available -> DOM is rendered happens in three discrete steps as well, so it’s hard to go directly from spinner to the desired DOM content without passing an undesired empty DOM state. This specific case I’m thinking of goes via for-by though, so maybe it’s related to the eagerness you mentioned previously @U09K620SG. I’m thinking it might be possible to fix both by sending back dom/node as a signal that everything is mounted and visible, but I haven’t tested yet. If it works, the second step will be to figure out how to do it so that it doesn’t result in a mess of spaghetti.

Is this demonstrated in the project repo you sent me?

Not quite, I removed the code when I couldn’t get it to work. I’ll add it back and ping you.

Please do, thanks. We are planning some changes here so I do not recommend you add hacks

and the e/for latency issue is involved

transactional dom writes is being considered

The DOM is very untransactional in its natural state. A lot of effort has gone into React to hide this fact, AFAIK.

we understand

Yeah, not to say that the solution that React delivers isn’t without its problems either. They chose to set the granularity at the component level, so it’ll provide some transactionality at the level of the component. The fact that Electric is more granular than that is a major strength in most cases, I would say.

react is also limited by the reconcilier which makes assumptions

i'm not the engineer who did the research so far so i dont want to comment too much, but we did do a discovery spike to see what other frameworks are doing

Thank you @U09K620SG. A lot of drive in this project makes me trust it more.

In terms of lists from e/watch, how does Electric handle rerendering? Suppose the watched variable goes from (a b c) to (a d b c e b). If there is a for loop over the variable, how does render/rerender happen?

I guess it's related to https://clojurians.slack.com/archives/C7Q9GSHFV/p1690889713294449?

yes, search for e/for-by

also review https://electric.hyperfiddle.net/user.demo-system-properties!SystemProperties

also https://clojureverse.org/t/electric-clojure-a-signals-dsl-for-fullstack-web-ui/9788/32?u=dustingetz

Really interesting. So a reactive database like https://github.com/mhuebert/re-db is not strictly necessary? Electric just takes care of the diffs in db watch naturally, and stops propagation when a higher node in the s-expr tree is equivalent.

I found myself being confused (then pleasantly surprised) when my console logs weren't being printed when I thought the "component" should re-render. It takes some time to get used to the fine-grained nature of things, and how things sort of "just work" at the compiler level.

But I guess re-db can be useful since with something like (e/server (some-query db)) will re-run every time !conn updates, whereas a reactive database can prevent that from happening. But at least upstream, this won't trigger an update if (some-query db) hasn't changed.

Yeah, it's great to have reactivity/streaming inside the data layer, and if you do have a streaming data layer, Electric can speak that natively

Unfortunately, knowing when to refresh your queries (for a data layer that is not streaming) is a hard problem, and it's not one that Electric solves out of the box

Most databases, e.g. Postgres and Datomic, provide a notification API so that you can understand what changed and use that information to know which queries to recheck – for many apps a simple heuristic will do. We can also help you throttle various queries as not all of your app needs to be realtime – in a scaled situation likely only certain restricted components, like a chat component, would be realtime, and that would be backed by a message queue or document store that offers efficient subscription

Thanks @U09K620SG, makes a lot of sense.

No need for multiple java runtimes. It should work exactly how you described it.

yes, (int s) compiles roughly to (m/latest int s)

Here is a pseudocode (for understanding) of the compiler transformation : https://github.com/hyperfiddle/electric/blob/f74a488e442c870b6a5409f45f49fd8a8a972240/src-docs/user/electric/electric_compiler.clj#L29-L31

That pseudo-transformation excludes the client/server network management

The real transformation is this: https://github.com/hyperfiddle/electric/blob/f74a488e442c870b6a5409f45f49fd8a8a972240/src-docs/user/electric/electric_compiler_internals.cljc#L27

the Electric compiler uses a real Clojure/Script analyser to transform clojure/script syntax into "DAG bytecode" for a "DAG virtual machine"

Electric is basically the JVM but for network

Fascinating

Don't be scared, this is why we waited 2.5 years to release anything publicly

it is robust

How similar would you say this is to the SSA/VM macro logic of core.logic or https://github.com/leonoel/cloroutine/blob/master/src/cloroutine/impl.cljc?

Leo wrote electric, missionary and cloroutine. With that said the electric VM is a VM of its own suited to solve the problems electric is meant to solve (a reactive network-transparent language)

That's good to know. I knew he wrote Missionary and Cloroutine, but I hadn't realized he was directly involved with Electric as well. 🙂

It's not SSA but cloroutine does implement a clojure/script analyzer and electric does too so they are similar in that aspect

fwiw cloroutine is not really on the stack in Electric, cloroutine is used to implement the missionary m/ap and m/cp syntax sugar macros but those are not essential to the Electric compiler transform

I weep for those not already familiar with Clojure and the cljs ecosystem. Learning all of this at once would be challenging to say the least. I've read the Missionary tutorial twice and it's still not sticking. 😒

FWIW, I don't think that new users in the future will need to know internal details like this. Writing "Electric code" is straightforward. It's just for those that need to be convinced of the correctness of the internals that need to dig into these details. 🙂

Hopefully. It's not yet clear to me yet what can be omitted.

Missionary docs need dramatic improvement, that's not on you

Leo is a walking computer science textbook

Electric also needs streamlining, we will do another pass this fall

e/for-by can. Note that dropping a heavy algorithm as-is into electric will degrade its performance. But if you use e/for-by in the right place(s) it will be incremental

Very interesting

by "heavy alg" we mean, say, code designed for linear execution, like a database query engine

Compare to reactive/async execution which is IO bound

use Clojure for linear code, use Electric for reactive code

I'm transforming timeseries data with a bunch of transducers. Each transducer adds its own field to the the timeseries, potentially using values from a previous transducers. For example:

{:t 1, :v 100}
{:t 2, :v 150}

;; After transducers 'plus5'

{:t 1, :v 100, :plus5 105}
{:t 2, :v 150, :plus5 155}

I would use Missionary if your algorithm is concurrent

Electric includes network management which is not free

Processing timeseries data sounds compute bound not IO bound

@U09FL65DK Can you elaborate on e/for-by? What's the difference (rendering-wise) with e/for?

My understanding is that for-by identifies the results with a 'key', similar to how it's done with components in Reagent. Assuming (for [x coll] ...), with current value of x being {:id 1 :msg "hello"} : In Reagent, we'd do something like this: ^{:key (:id x)} [:div (:msg x)]]

(e/for ...) is just (e/for-by identity ...)

honestly we'll probably remove e/for , the only benefit is that the early tutorial code is slightly cuter (can delay the explanation)

That makes a lot of sense. In terms of internals, what happens if there are two elements with the same key?

> In terms of internals, what happens if there are two elements with the same key? The diff algorithm will choose the closest, according to position in sequence

dom/div and friends mount a new DOM node as a side effect

this is what I mean by effectful rendering in the London talk

the effects are supervised by a functional effect system which provides strong guarantees wrt effectful evaluation that imperative languages do not

• cancellation • concurrency and backpressure combinators • composability / referential transparency – coordinate large scale effect fabrics • glitch-free event propagation • correct error handling by default (process supervision) • resource lifecycle (RAII) – strong resource cleanup guarantees

See the talk for more discussion of this

I already watched your talk, but a bunch of stuff flew over my head because I lacked the knowledge/context to appreciate it.

I'll make sure to re-watch it once I'm more versed in the subject.

Ah ok! TY for the feedback

electric code needs to stay in sync on the server and client side, which means the whole ns is often reloaded

electric uses a custom hot code reloading strategy: we hacked shadow compilation so that whenever a cljc namespace with electric code in it is recompiled, the server namespace is simultaneously reloaded

this prevents spooky undefined behavior when a fresh client DAG connects to a stale server DAG

Makes sense, it's just unexpected when coming from... well, the rest of the Clojure world. 😉

yes :)

have you seen https://hyperfiddle.github.io/#/page/electric%20internal%20architecture

Yeah, great doc

does two windows solve your problem?

Here is the tutorial repo, is that what you're looking for? https://github.com/hyperfiddle/electric-examples-app

Yes, tyty.

E.g.

(defn update-js-array [_ js-array]
  (.push js-array #js {})
  js-array)
(e/defn Example [input]
  (let [js-array (array)
        updated (update-js-array input js-array)]
    (js/console.log (count updated) updated input)))

Yes, the reactive evaluation uses clojure.core/= to decide if it can skip work

a dumb hack could be to wrap the returned array in a unique wrapper object

JS wrapper object you mean? Otherwise it seems like it'd be the same issue

yes

Gotcha; I hadn't considered that. I only tried a CLJ wrapper. 🙂 Good enough for me without dropping down to some sort of missionary flow workaround 🤷

you can also cachebust with a counter (assuming you control all reference writers)

That's kind of how I'm doing it now, though having to keep the JS array and its associated cachebuster together and used together everywhere isn't quite as clean as just wrapping in a new JS object.

(defn ->Object "allocate a process-unique identity that cannot collide" []
  #?(:clj (Object.) :cljs (js/Object.)))

{(->Object) ref} i think would work, you just have to remember to pass around the composite and remember to unbox only from Clojure

Yes you get Clojure semantics not Electric. Same if you use clojure.core/fn inside an Electric function – you get clojure semantics inside the clojure function

electric functions can call clojure functions; clojure functions cannot call electric functions

magic network only works in electric functions

Oh I see, so I can't use e/server inside a defn ?

correct

Yes, you can 🙂 (F. v) just desugars to (new F v); doesn't matter as long as F is bound to an Electric function

Great, thanks