Fork me on GitHub
#hyperfiddle
<
2023-02-13
>
tatut05:02:29

tried to todo example, feels like magic

🙂 1
tatut06:02:43

the readme mentions network planner improvements, is that related to the chattiness of the WS connection? it looks that currently clicking a checkbox in a todo item sends 40 messages and 6kb of data. It works very well locally.

xificurC09:02:00

Yes, it is related. Note that we routinely run the demos and other apps under high latency and the ~40 messages don't lag as much as you might expect

Dustin Getz13:02:57

Yes the network planner today has (1) a major implementation flaw (not design flaw) that has not been a priority to fix, and (2) most of the traffic is streaming (like Netflix), not request/response – the two sites are just keeping each other informed. We haven't actually implemented the optimizer, we currently just broadcast all control flow switches (which ends up being much more than is actually needed).

tatut06:02:48

I tried to see how it breaks,

(time (doseq [i (range 1000)] (d/transact! !conn [{:task/description (str "FOO " i) :task/status :active}])))
in the REPL failed after 38 seconds with pong timeout and null pointer exception… the UI looked weird, slowly getting new entries with checkbox first and the text later

tatut06:02:36

I guess in a “real” app you wouldn’t watch a source that changes so frequently

Dustin Getz12:02:23

thanks, it's almost certainly a bug, we will take a look

xificurC13:02:11

I took a look at this and largely couldn't reproduce. Maybe my machine is beefier? Nevertheless there is room for improvement here, it takes a while to load/render and exhibits high cpu/gc and memory load. The behavior is also different in chrome and firefox. Now that the groundwork is laid out we'll have more time to work on performance and other optimizations.

Dustin Getz14:02:34

@U11SJ6Q0K did you have the chrome dev tools open?

Dustin Getz14:02:12

the "UI weirdness" is an artifact of the naive streaming render model, we orchestrate point effects as fast as possible and naively today (we will deal with this). So the visual artifact likely shows that something is being backpressured here

Dustin Getz14:02:36

Also note that each transact causes the datascript query to rerun, which is likely a substantial portion of the resource consumption

tatut14:02:31

I most likely did have devtools open, I almost always have it open

Dustin Getz14:02:08

The chrome devtools don't play that well with our large macroexpansion (we need to optimize it). It's mostly fine but for stress tests like you just did, the debugger instrumentation is likely a huge factor in performance

tatut14:02:51

could be the same issue with my openlayers test, which seemed to work much better in safari

👍 2
Dustin Getz14:02:51

I think the crash you saw is the client missed a heartbeat due to the extreme backpressure

Dustin Getz14:02:28

I dont think we lost any state, the app would have correctly settled in the end

tatut14:02:07

yes, a busy loop writing transactions is not a likely scenario

tatut14:02:35

I’ll play around more and try without devtools open

Dustin Getz14:02:48

you want to do a zoom? would be happy to chat and you know, “zoom coffee” or whatever

tatut14:02:05

don’t have time now, but thanks for the offer

👍 2
tatut15:02:49

had a while to try it again

app.todo-list> (time (doseq [i (range 1000)] (d/transact! !conn [{:task/description (str "FOO " i) :task/status :active}])))
ERROR hyperfiddle.electric-jetty-adapter: Websocket handler failure #error {
 :cause Websocket pong timeout.
 :data {}
 :via
 [{:type clojure.lang.ExceptionInfo
   :message Websocket pong timeout.
   :data {}
   :at [hyperfiddle.electric_jetty_adapter$make_heartbeat$cr19868_block_4__19878 invoke electric_jetty_adapter.clj 23]}]
 :trace
 [[hyperfiddle.electric_jetty_adapter$make_heartbeat$cr19868_block_4__19878 invoke electric_jetty_adapter.clj 23]
  [cloroutine.impl$coroutine$fn__15076 invoke impl.cljc 60]
  [missionary.impl.Sequential step Sequential.java 86]
  [missionary.impl.Sequential$1 invoke Sequential.java 109]
  [missionary.impl$absolver$fn__14964 invoke impl.clj 15]
  [missionary.impl.RaceJoin terminated RaceJoin.java 33]
  [missionary.impl.RaceJoin$1 invoke RaceJoin.java 52]
  [missionary.impl.RaceJoin$2 invoke RaceJoin.java 67]
  [missionary.impl.Sleep$Scheduler trigger Sleep.java 60]
  [missionary.impl.Sleep$Scheduler run Sleep.java 74]]}
ERROR hyperfiddle.electric-jetty-adapter: Websocket error #error {
 :cause Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
 :via
 [{:type java.lang.NullPointerException
   :message Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.RT.aget(Object[], int)" is null
   :at [hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_text__19919 invoke electric_jetty_adapter.clj 77]}]
 :trace
 [[hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_text__19919 invoke electric_jetty_adapter.clj 77]
  [ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__809 invoke websocket.clj 155]
  [ring.adapter.jetty9.websocket.proxy$org.eclipse.jetty.websocket.api.WebSocketAdapter$WebSocketPingPongListener$12d400b6 onWebSocketText nil -1]
  [org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver onTextMessage JettyListenerEventDriver.java 296]
  [org.eclipse.jetty.websocket.common.message.SimpleTextMessage messageComplete SimpleTextMessage.java 69]
  [org.eclipse.jetty.websocket.common.events.AbstractEventDriver appendMessage AbstractEventDriver.java 67]
  [org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver onTextFrame JettyListenerEventDriver.java 235]
  [org.eclipse.jetty.websocket.common.events.AbstractEventDriver incomingFrame AbstractEventDriver.java 152]
  [org.eclipse.jetty.websocket.common.WebSocketSession incomingFrame WebSocketSession.java 326]
  [org.eclipse.jetty.websocket.common.extensions.AbstractExtension nextIncomingFrame AbstractExtension.java 148]
  [org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension nextIncomingFrame PerMessageDeflateExtension.java 111]
  [org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension forwardIncoming CompressExtension.java 169]
  [org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension incomingFrame PerMessageDeflateExtension.java 90]
  [org.eclipse.jetty.websocket.common.extensions.ExtensionStack incomingFrame ExtensionStack.java 202]
  [org.eclipse.jetty.websocket.common.Parser notifyFrame Parser.java 225]
  [org.eclipse.jetty.websocket.common.Parser parseSingleFrame Parser.java 259]
  [org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection onFillable AbstractWebSocketConnection.java 459]
  [org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection onFillable AbstractWebSocketConnection.java 440]
  [org.eclipse.jetty.io.AbstractConnection$ReadCallback succeeded AbstractConnection.java 311]
  [org.eclipse.jetty.io.FillInterest fillable FillInterest.java 105]
  [org.eclipse.jetty.io.ChannelEndPoint$1 run ChannelEndPoint.java 104]
  [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill runTask EatWhatYouKill.java 336]
  [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill doProduce EatWhatYouKill.java 313]
  [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill tryProduce EatWhatYouKill.java 171]
  [org.eclipse.jetty.util.thread.strategy.EatWhatYouKill run EatWhatYouKill.java 129]
  [org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread run ReservedThreadExecutor.java 383]
  [org.eclipse.jetty.util.thread.QueuedThreadPool runJob QueuedThreadPool.java 882]
  [org.eclipse.jetty.util.thread.QueuedThreadPool$Runner run QueuedThreadPool.java 1036]
  [java.lang.Thread run Thread.java 833]]}
"Elapsed time: 42797.571791 msecs"
this time only safari connected (no devtools open), still happens

👍 2
tatut15:02:12

data was ok, all items are there as expected after reloading the page

tatut15:02:32

on a Macbook Pro M1 fwiw

tatut15:02:35

app.todo-list> (select-keys (System/getProperties) ["java.vendor" "java.version"])
{"java.vendor" "Azul Systems, Inc.", "java.version" "17.0.3"}

borkdude11:02:12

I posted it on HN, it's on the first page now 😅

👍 9
🔥 7
🚀 1
Dustin Getz17:02:36

thank you 🙂

borkdude12:02:08

Since there was no "About" in the electric repo, I chose the title "Electric Clojure: unified fullstack application compiler

tatut13:02:24

it looks like the dom element macro should work with custom elements, so you could use electric to drive some js library that is wrapepd in webcomponent custom elements

Dustin Getz14:02:32

Yeah! the dom wrapper is nothing more than point writes + Electric signals + 300 LOC of macros for declarative syntax. All the heavy lifting is done by Missionary. That means we can drive Swing GUIs, terminal GUIs, anything

👍 2
tatut14:02:30

sounds great, I’ll try to play around with it

adamfrey16:02:48

@dustingetz very excited to hear that. I'm interested if it could be combined with #CVB8K7V50’s model for agnostic platform UI rendering

👀 1
tatut17:02:49

https://twitter.com/tatut/status/1625192601354641408 seems to work somewhat… my first experiment with electric

👏 2
❤️ 2
tatut18:02:02

I just took an openlayers wrapped as web components and used that, don’t know if the vector layer update is meant to be used that way (so the bugs seen in the video might be just me misusing the web component)

tatut18:02:34

hmm, it actually seems to work much better in safari than chrome

Dustin Getz18:02:19

If you're using cursive, set Default to Only Indent

Dustin Getz18:02:14

Would love if you shared this in the HN and r/clojure threads

vlaaad17:02:17

Congrats on the release @dustingetz! I’ll try to find some time to play with it soon

5
🙂 1