Fork me on GitHub

tried to todo example, feels like magic

🙂 1

the readme mentions network planner improvements, is that related to the chattiness of the WS connection? it looks that currently clicking a checkbox in a todo item sends 40 messages and 6kb of data. It works very well locally.


Yes, it is related. Note that we routinely run the demos and other apps under high latency and the ~40 messages don't lag as much as you might expect

Dustin Getz13:02:57

Yes the network planner today has (1) a major implementation flaw (not design flaw) that has not been a priority to fix, and (2) most of the traffic is streaming (like Netflix), not request/response – the two sites are just keeping each other informed. We haven't actually implemented the optimizer, we currently just broadcast all control flow switches (which ends up being much more than is actually needed).


I tried to see how it breaks,

(time (doseq [i (range 1000)] (d/transact! !conn [{:task/description (str "FOO " i) :task/status :active}])))
in the REPL failed after 38 seconds with pong timeout and null pointer exception… the UI looked weird, slowly getting new entries with checkbox first and the text later


I guess in a “real” app you wouldn’t watch a source that changes so frequently

Dustin Getz12:02:23

thanks, it's almost certainly a bug, we will take a look


I took a look at this and largely couldn't reproduce. Maybe my machine is beefier? Nevertheless there is room for improvement here, it takes a while to load/render and exhibits high cpu/gc and memory load. The behavior is also different in chrome and firefox. Now that the groundwork is laid out we'll have more time to work on performance and other optimizations.

Dustin Getz14:02:34

@U11SJ6Q0K did you have the chrome dev tools open?

Dustin Getz14:02:12

the "UI weirdness" is an artifact of the naive streaming render model, we orchestrate point effects as fast as possible and naively today (we will deal with this). So the visual artifact likely shows that something is being backpressured here

Dustin Getz14:02:36

Also note that each transact causes the datascript query to rerun, which is likely a substantial portion of the resource consumption


I most likely did have devtools open, I almost always have it open

Dustin Getz14:02:08

The chrome devtools don't play that well with our large macroexpansion (we need to optimize it). It's mostly fine but for stress tests like you just did, the debugger instrumentation is likely a huge factor in performance


could be the same issue with my openlayers test, which seemed to work much better in safari

👍 2
Dustin Getz14:02:51

I think the crash you saw is the client missed a heartbeat due to the extreme backpressure

Dustin Getz14:02:28

I dont think we lost any state, the app would have correctly settled in the end


yes, a busy loop writing transactions is not a likely scenario


I’ll play around more and try without devtools open

Dustin Getz14:02:48

you want to do a zoom? would be happy to chat and you know, “zoom coffee” or whatever


don’t have time now, but thanks for the offer

👍 2

had a while to try it again

app.todo-list> (time (doseq [i (range 1000)] (d/transact! !conn [{:task/description (str "FOO " i) :task/status :active}])))
ERROR hyperfiddle.electric-jetty-adapter: Websocket handler failure #error {
 :cause Websocket pong timeout.
 :data {}
 [{:type clojure.lang.ExceptionInfo
   :message Websocket pong timeout.
   :data {}
   :at [hyperfiddle.electric_jetty_adapter$make_heartbeat$cr19868_block_4__19878 invoke electric_jetty_adapter.clj 23]}]
 [[hyperfiddle.electric_jetty_adapter$make_heartbeat$cr19868_block_4__19878 invoke electric_jetty_adapter.clj 23]
  [cloroutine.impl$coroutine$fn__15076 invoke impl.cljc 60]
  [missionary.impl.Sequential step 86]
  [missionary.impl.Sequential$1 invoke 109]
  [missionary.impl$absolver$fn__14964 invoke impl.clj 15]
  [missionary.impl.RaceJoin terminated 33]
  [missionary.impl.RaceJoin$1 invoke 52]
  [missionary.impl.RaceJoin$2 invoke 67]
  [missionary.impl.Sleep$Scheduler trigger 60]
  [missionary.impl.Sleep$Scheduler run 74]]}
ERROR hyperfiddle.electric-jetty-adapter: Websocket error #error {
 :cause Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
 [{:type java.lang.NullPointerException
   :message Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
   :at [hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_text__19919 invoke electric_jetty_adapter.clj 77]}]
 [[hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_text__19919 invoke electric_jetty_adapter.clj 77]
  [ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__809 invoke websocket.clj 155]
  [ring.adapter.jetty9.websocket.proxy$org.eclipse.jetty.websocket.api.WebSocketAdapter$WebSocketPingPongListener$12d400b6 onWebSocketText nil -1]
  [ onTextMessage 296]
  [org.eclipse.jetty.websocket.common.message.SimpleTextMessage messageComplete 69]
  [ appendMessage 67]
  [ onTextFrame 235]
  [ incomingFrame 152]
  [org.eclipse.jetty.websocket.common.WebSocketSession incomingFrame 326]
  [org.eclipse.jetty.websocket.common.extensions.AbstractExtension nextIncomingFrame 148]
  [org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension nextIncomingFrame 111]
  [org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension forwardIncoming 169]
  [org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension incomingFrame 90]
  [org.eclipse.jetty.websocket.common.extensions.ExtensionStack incomingFrame 202]
  [org.eclipse.jetty.websocket.common.Parser notifyFrame 225]
  [org.eclipse.jetty.websocket.common.Parser parseSingleFrame 259]
  [ onFillable 459]
  [ onFillable 440]
  [$ReadCallback succeeded 311]
  [ fillable 105]
  [$1 run 104]
  [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill runTask 336]
  [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill doProduce 313]
  [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill tryProduce 171]
  [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill run 129]
  [org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread run 383]
  [org.eclipse.jetty.util.thread.QueuedThreadPool runJob 882]
  [org.eclipse.jetty.util.thread.QueuedThreadPool$Runner run 1036]
  [java.lang.Thread run 833]]}
"Elapsed time: 42797.571791 msecs"
this time only safari connected (no devtools open), still happens

👍 2

data was ok, all items are there as expected after reloading the page


on a Macbook Pro M1 fwiw


app.todo-list> (select-keys (System/getProperties) ["java.vendor" "java.version"])
{"java.vendor" "Azul Systems, Inc.", "java.version" "17.0.3"}


I posted it on HN, it's on the first page now 😅

👍 9
🔥 7
🚀 1
Dustin Getz17:02:36

thank you 🙂


Since there was no "About" in the electric repo, I chose the title "Electric Clojure: unified fullstack application compiler


it looks like the dom element macro should work with custom elements, so you could use electric to drive some js library that is wrapepd in webcomponent custom elements

Dustin Getz14:02:32

Yeah! the dom wrapper is nothing more than point writes + Electric signals + 300 LOC of macros for declarative syntax. All the heavy lifting is done by Missionary. That means we can drive Swing GUIs, terminal GUIs, anything

👍 2

sounds great, I’ll try to play around with it


@dustingetz very excited to hear that. I'm interested if it could be combined with #CVB8K7V50’s model for agnostic platform UI rendering

👀 1
tatut17:02:49 seems to work somewhat… my first experiment with electric

👏 2
❤️ 2

I just took an openlayers wrapped as web components and used that, don’t know if the vector layer update is meant to be used that way (so the bugs seen in the video might be just me misusing the web component)


hmm, it actually seems to work much better in safari than chrome

Dustin Getz18:02:19

If you're using cursive, set Default to Only Indent

Dustin Getz18:02:14

Would love if you shared this in the HN and r/clojure threads


Congrats on the release @dustingetz! I’ll try to find some time to play with it soon

🙂 1