Fork me on GitHub
#hyperfiddle
<
2024-02-11
>
simonkatz23:02:45

I would like to display a table that’s ~50 columns wide and several thousand rows high. The code below is a simplified version of what I’d like to do. It displays a times table, with the max x and y values as input. The reverses are just so I can easily see when things change. The code I have is slow to render when I get to a few hundred rows, and things break (“Aw, Snap!” in Chrome) when I try a few thousand rows. Is there anything I can do to make this work better? (Maybe I’m doing something wrong. Maybe it’s not wrong as such but more sophistication is needed.)

simonkatz23:02:35

It’s lines 30-36 above that are the pertinent ones. The rest is just dealing with the two input fields.

wei01:02:53

i also had performance issue with a long table, and was recommended to look into virtual scroll https://clojurians.slack.com/archives/C7Q9GSHFV/p1706795492692999?thread_ts=1706623472.727089&amp;cid=C7Q9GSHFV

Dustin Getz01:02:30

we virtualized our grids from day 1 and never really looked back, we have a demo somewhere of a folder tree view over node_modules, 10k-20k rows perfect performance Philadelphia to Paris

👍 1
Dustin Getz01:02:30

electric is not a fast language in the way that rust is fast, it is fast inclusive of IO and compared to hand coded IO and client databases and such, but as a language it pays a heavy runtime cost (today at least). you definitely do not want to be processing large collections in electric. if you are ever forced to do that you should drop down to clojure

👍 2
simonkatz10:02:36

Thanks; I’ll take a look at the Datomic browser.

simonkatz10:02:15

How would I “drop down to clojure” in my example? I’ve tried this:

(defn table [max-x max-y]
  (dom/table
    (dom/props {:style {:white-space "nowrap"}})
    (dom/tbody
      (for [i (reverse (map inc (range max-y)))]
        (dom/tr
          (for [j (reverse (map inc (range max-x)))]
            (dom/td (dom/text (* i j)))))))))
When I refresh my browser the table appears briefly, but then disappears and there’s an "Index 0 out of bounds for length 0" error.

Dustin Getz12:02:00

it’s not legal to put electric code in clojure.core/for, which presumably wraps the body continuation in a clojure lambda

Dustin Getz12:02:40

i’m not sure why it caused a desync, i’ll log a ticket to take a look

simonkatz12:02:22

FWIW, I’m not getting the index out of bounds error anymore (in a fresh REPL). I’m getting a more reasonable No such var: hyperfiddle.electric-dom2/new-node. So maybe previously things had gotten into a weird state.

simonkatz12:02:53

So, for my example do I need to be looking at the virtualization thing? And was the “drop down to clojure” for processing large collections comment something more general?

Dustin Getz13:02:59

generally, if you want to process 1000s of items in a loop (and do this every time something changes) you should leave electric and do it in clojure/script using defn. If you truly need to have 1000s of dom elements on the screen, electric v2 is not useful for that, you can use document.createElement etc from clojurescript, or try using a react wrapper if they are able to do it (vdom approaches have the same linear scan scalability issues but perhaps they are optimized enough that they can handle 1000 idk). Electric v3 however actually will be able to manage 1000s of dom elements efficiently without linear scans due to the differential dataflow internals

Dustin Getz13:02:34

The real answer is virtual scroll, I don't understand why anyone would ever want to have 1000s of dom elements on the page and especially in one monster collection, you start exceeding browser performance thresholds

simonkatz13:02:56

OK; thank you. That’s all very helpful.