Fork me on GitHub
#hyperfiddle
<
2023-02-15
>
Omar02:02:10

I've noticed on the starter app and in my own project once the electric server goes down that the UI will disappear. How should I manage this? There a way to display anything when it goes down (perhaps even client data?) and execute something when the connection is re-established?

Geoffrey Gaillard09:02:39

An electric clojure program is split in two parts (client and server) at compile time. Both programs are run on their respective peer. They run at the same time and they always go together. The electric server is not supposed to go down unless there is a network issue or the client gracefully shutdowns (navigating away) In case of network instability, the client peer will reconnect on itss own and reboot the server side. Because this is functional programming, rebooting the server side with the client state should just restore the page. The https://github.com/hyperfiddle/electric/blob/master/src/hyperfiddle/electric_client.cljs has a connection lifecycle, but it is not exposed to userland at the moment. Could you tell me more about your use case?

Omar10:02:45

Yeah! I think being able to still render something to indicate to the user that the app will be back/lost connection instead of going completely blank, then triggering a notification to let them know it's back. If it just completely goes blank that will just be off putting to most people and they'll navigate away from the site.

Geoffrey Gaillard10:02:54

Understood. I’ll record this in our issue tracker. In the meantime, one potential solution to mitigate this would be to use CSS. Let me explain: • your electric clojure program mounts into some https://github.com/hyperfiddle/electric/blob/fee5c441ad809def70b17ce4f7b7c537f3a37a09/src-dev/user_main.cljc#L96 (similar to ReactDOM.createRoot(getElementById(…))) • When the program starts, it will add children dom nodes to the root element • when the electric program shuts down, all elements added by electric clojure will be removed • let’s suppose you root element is <div id="my-program-root"></div> • you can add, just after it <div id="my-fallback-message"> Connection lost. Reconnecting… (or please refresh the page) </div> • CSS: #my-program-root:empty + #my-fallback-message {display:none;} . The fallback message will be visible only when the program root is empty.

J09:02:09

Hi guys! Thanks for this amazing lib. I was excited since I saw the hydradboi talk. So again congrats! Now, I play with it and I’m wonder how integrate sql backend. I test with this code:

(e/defn AddPokemon [!pokemons]
  (e/client
   (InputSubmit. (e/fn [v]
                   (e/server
                    (e/discard
                     (swap! !pokemons conj (sql/insert-pokemon pg-conn {:id (random-uuid) :display_name v}))))))))
(e/defn App []
  (e/server
   (binding [pg-conn (sql/create-connection)]
     (let [!pokemons (atom (sql/list-pokemon pg-conn))
           pokemons (e/watch !pokemons)]
       (e/client
        (dom/h1 (dom/text "Pokemons"))
        (AddPokemon. !pokemons)
        (dom/div
         (e/for-by :pokemon/id [{:keys [pokemon/display_name]} (e/server pokemons)]
                   (e/client
                    (dom/p (dom/text display_name))))))))))
I get the InputSubmit button of the todo demo and I wanted to refresh the list of pokemons when a new pokemon is added to the database. But my code is incorrect because I passe !pokemons server atom directly into the AddPokemon component. How can I achieve this without create a global ref to the !pokemons atoms?

👀 4
Geoffrey Gaillard10:02:07

If I understand correctly, it looks like you are trying to manually maintain a list of pokemons in the !pokemons atom. You initialize the atom with a query, then you manually add new pokemons to it. You cannot ensure !pokemons will stay in sync with the database. I’ll get back to you with a code snippet

xificurC10:02:59

Note that while your code isn't operationally correct (you won't see updates to the list of pokemons done outside from your application) semantically it is OK and will run in the future. !pokemons is passed on to a function running on the client but it never uses it, so the transfer can be optimized away.

👍 2
J16:02:25

Thanks a lot @U2DART3HA

Geoffrey Gaillard16:02:19

I’m polishing the SQL snippet a bit more, will share again soon

J06:02:44

Thanks @U09K620SG ! I will take a look! 🙏

J08:02:50

(e/defn AddPokemon []
  (e/client
    (InputSubmit. (e/fn [v] 
                    (e/server
                      (try
                        (let [x (sql/insert-pokemon pg-conn {:id (random-uuid) :display_name v})]
                          (swap! !dirty inc) ; success
                          x)
                        (catch SQLException e
                          ; handle it, or alternatively let it propagate upward
                          (e/client (dom/props {:class "error" :style {:background-color "red"}})))))))))
Don’t need to use e/wrap on the sql/insert-pokemon call?

Geoffrey Gaillard08:02:19

Good catch. Yes it is needed. All IO blocking operations should be wrapped.

👍 2
J09:02:58

I have a concern about this:

(binding [pg-conn (sql/create-connection)
            !dirty (atom 0)] ; make available anywhere that pg-conn is available
        (let [pokemons (e/wrap (sql/list-pokemon pg-conn (e/watch !dirty)))] ; reruns when dirty changes
It’s normal that (e/watch !dirty)is inside the (e/wrap) and inside the (sql/list-pokemon)? :thinking_face:

Geoffrey Gaillard10:02:26

Your concern is valid. There’s a mistake in the code. Sorry for the confusion. Electric Clojure compiles your code into a graph. Values propagates through the graph (see below) • (sql/list-pokemon pg-conn (e/watch !dirty)) is valid. • (e/wrap … (e/watch !dirty)) is not valid. e/wrap is a macro taking regular Clojure code. e/watch is a macro emitting Electric Clojure code. So e/watch cannot be used in e/wrap. This is correct:

(let [dirty (e/watch !dirty)]
  (e/wrap (sql/list-pokemon pg-conn dirty))

J11:02:23

Thanks @U2DART3HA and @U09K620SG it works!

❤️ 4
vlaaad10:02:03

would be cool to get some sort of protocol defined, so e.g. there is a guide on how to swap compatible servers (is it only a websocket endpoint that matters?), databases (sql/datomic/file on disc), clients (TUI, javafx etc)

👀 2
xificurC10:02:13

the client/server messaging protocol happens to be websocket for the web. In our tests we run the client and server peer in the same process, therefore the peers communicate in process. In the end the peers require 1 fn for read and 1 fn for write, so adding new protocols isn't too hard. We will discuss different databases in the coming weeks, e.g. see the thread above with Postgres. The "hard" part is making the DB notify us there's new data so we should re-run our queries. We discussed adding a swing demo but didn't find enough time to get to it. The dom library is 300loc, of which 100 is wrapping all the native dom element types in macros, so the whole impl is ~200loc. So adding new UIs also won't be too hard.

👍 2
Dustin Getz14:02:18

@U47G49KHQ in addition to what @U09FL65DK said, Jetty is the only server Electric supports today – see electric-jetty-server and electric-jetty-adapter if you want to port them to other servers

Dustin Getz14:02:37

Databases - we're looking into a Postgres demo, we had Datomic and XTDB as well but they rotted and need to be restored

reedho08:02:15

I think using with Datalevin is gone be simpler?

reedho08:02:01

I would like to port adapter for httpkit as an excercise.

👀 2
simongray10:02:00

I am wondering how you are going to deal with app hydration in Electric Clojure? Is this something you're working towards...? It would be the holy grail to have an implicitly hydrated full-stack app. i.e. something like a (e/hydrated ...) scope that runs on server or client depending on the hydration state.

👍 2
Geoffrey Gaillard11:02:28

You mean hydrated as in resuming SSR on the client?

simongray11:02:09

Basically, let's say we replace every instance of e/client with a hypothetical e/hydrated . These scopes would run entirely on the server when the initial HTML page is generated and entirely on the client after the initial render in the browser.

Geoffrey Gaillard11:02:31

Oh yes, definitely. We have the electric-dom library writing to the browser DOM. Adapting it to a server-side DOM impl and emitting a stream of static markup could be an exercise for the reader. Rerunning on the client and correlating SSR generated node ids is doable too. But it’s not clear to me if hydration would be an actual perf improvement in a model like electric clojure.

tatut11:02:54

example, if you add a thousand todo items in the sample app, the page load get quite slow, you see a white page for a long time

tatut11:02:39

and then once the websocket receives a huge payload, does rendering begin… similar to SPA apps, I could envision initial SSR showing a big speedup in those cases where you have lots of dynamic data on a page

Dustin Getz11:02:29

i was talking to ryansolid about this, iiuc ssr perf, when the browser is fast, is about eliminating the extra round trip to bootstrap the app from the client. can write more later, also i am not an expert at this

Dustin Getz11:02:06

the perf hit from your test of adding 1000 todo items is not a “fair” test also, i think it is churning queries and datascript also likely is not responding to cancellation signals, multiple layers are doing bad stuff that needs to be fixed

tatut11:02:07

I might be biased in favor of SSR as my ripley library is all about that

👀 4
Geoffrey Gaillard11:02:39

"once the websocket receives a huge payload". Electric clojure enables you to avoid this issue entirely. Use e/for on the server to diff your collection, and paginate on the server too. In an production setup, only what's visible to client (in the viewport) should transfer.

tatut11:02:35

yes, agreed… but the browser is super fast at parsing and showing HTML. A thousand todo items wouldn’t break a sweat

👍 2
tatut11:02:57

so there are tradeoffs to server vs client rendering

Geoffrey Gaillard11:02:09

Yes definitely. We will need to experiment and measure.

xificurC13:02:31

How about no hydration, like https://qwik.builder.io/ is doing? Lazy loading the interactive parts as necessary. The core got mature enough for a public release but we haven't gone through the optimization phase yet. The performance considerations you have in mind will be addressed. Thanks for pushing the limits and thinking of ways to improve!

simongray13:02:37

To me it's not just about performance, it's about accessibility. An HTML page where the content is entirely JS-driven is effectively dead in a lot of contexts, i.e. anything that doesn't involve a JS-enabled, modern browser with a direct server connection. Simple SPAs are fine for "apps", but hydrated SPAs can also be used for "websites".

👀 4
👍 4
2
simongray13:02:27

an entirely JS-driven SPA is ephemeral so it doesn't work for what I would call websites, it is relegated to apps

👍 2
simongray13:02:35

a hydrated SPA can straddle both kingdoms, i.e you can have a traditional website that can be properly cached and will be a part of history (e.g. in the Wayback Machine) but it can also be highly interactive/dynamic on the client.

👍 2
simongray13:02:35

Personally, having no hydration means I would only ever consider using it for real-time web apps, never for anything website-like. It is still an awesome piece of engineering (seriously awesome) but that definitely limits its scope for me.

👀 2
🙂 2
xificurC13:02:37

which makes sense, we've been focused on building web apps. I logged a ticket in our issue tracker with your comments and we'll think about solutions

🙏 2
👍 2
simongray13:02:36

It's totally fine if this is out of scope, I think I get what you're trying to make here. A boy can dream, though. 😉

🙂 2
xificurC14:02:05

we're driven by our users' needs so of course we want to make their dreams come true 🙂 I'm not saying this will be done tomorrow of course, but I also don't think it's a harder problem than the ones we've already tackled, so there's no reason this couldn't be done some day

👍 2
Dustin Getz14:02:55

I want to be clear that Electric Clojure is for ultra-dynamic web applications, not websites. Next.js is excellent at websites and is getting better every month, the new Vercel Cloud Edge Functions stuff seems quite excellent, we will not be able to compete with that

👍 2
Dustin Getz14:02:06

Basically we need to focus our energies on making Electric world-class in one domain

Dustin Getz14:02:26

Certainly the Electric model can be extended to hybrid approaches, perhaps it's even easy to get a POC working, however matching Vercel's performance numbers is likely not within reach without similar amounts of capital

Dustin Getz14:02:41

Vercel raised $313M

Dustin Getz14:02:12

@U11SJ6Q0K I checked the 1000 todo items perf, I changed it to create 1000 records in one transaction to focus the microbenchmark on the rendering. Electric renders the 1000 in 4 seconds. Indeed, that is far too slow and should be optimized. We have a 10k dom elements benchmark internally which also reveals the perf issue here. My intuition is that the dom point effects are not actually the bottleneck. We need to profile, get real numbers, do a proper discovery as to what's actually happening

👍 2
Dustin Getz14:02:30

@U11SJ6Q0K Do you have any sources that benchmark rendering HTML from document vs rendering HTML from DOM (document.createElement)? There are a lot of confounding factors to eliminate. At the lowest level, both seem to do the same thing: receive a stream of text over network, parse the text, translate the instructions into DOM mutations

tatut14:02:33

I don’t remember anything recent, those might change as browser engines develop

tatut14:02:19

ripley just sets server rendered raw HTML as elt.outerHTML = … to bypass JS

Dustin Getz14:02:20

Is it still faster? React.js is 10 years old now, browsers have optimized for it

Dustin Getz14:02:28

also, event handlers etc

tatut14:02:45

I don’t honestly know

tatut14:02:56

what the situation is with current engines

Dustin Getz14:02:02

gotcha me either

Omar18:02:55

Personally for me SSR isn't important. If speed is the concern you can get around this by doing all you can to optimize your JS build size and getting it on a proper CDN. Before I've also kept my bootstrap data mandatory for an initial render to populate the views embedded as JSON on the html the backend spits out that can be parsed immediately so there isn't an additional API call to fill in your views after the JS executes. Then, when you're intelligent about prefetching things it outperforms any non spa.

pez11:02:23

Hot-reloading is a bit less surgical than I am used to. The web page seems to get an empty state and then re-populates. (No idea if this is what is happening, just get that feeling.) Is this something that can be smoothen out in the Electric future?

👍 2
Geoffrey Gaillard11:02:47

You got it right. Today the electric compiler performs full program analysis. So the whole electric clojure code is recompiled and the client reboots on hot reload to ensure client and server are in sync. Optimizing this is on our roadmap.

metal 8
6
👍 2
gratitude 1
Dustin Getz14:02:52

All: I might do a group onboarding zoom call, please indicate if interested

1️⃣ 32
dumrat14:02:30

Interested

👍 2
jmayaalv13:02:01

interested!

Dustin Getz19:02:10

First group onboarding tomorrow, Friday, 3pm - 5pm ET. Other timezones will be accommodated next week. Here's the deal: you have to have your video camera on. I'm starting with 4 people and seeing how it goes. First 4 people to DM me their email address are in and will get the zoom link.

Dustin Getz19:02:17

Next week will do a PT afternoon and a CET afternoon

2
👍 8
Dustin Getz19:02:08

Reply here or DM me if you need an asia-friendly TZ and we will try to figure it out

Panel23:02:52

Hi from Australia

👀 2
2
dumrat02:02:07

I'm from Singapore

👀 2
Dustin Getz12:02:00

Today is filled, more next week!

Dustin Getz16:02:38

Group zoom onboardings this week: • PT (San Fransisco) this Thurs Feb 23, 3pm - 5pm • CET (Paris) this Weds Feb 22, 3pm - 5pm • SGT (Singapore) this Friday Feb 24, 6pm - 8pm Remember, we do 1:1 onboardings as well which are more effective, please DM me for a 1:1! On our side, the 1:1s help us figure out how to explain this better and discover hidden problems, so we appreciate if you are able to make time for a 1:1. (You can also do both.) @dumrat @kjw0323 @zenflowapp @chromalchemy @v @damir.bijuklic @mattias @sfyire @tomisme @jmayaalv @sr @jaapmaaskant

👍 2
Dustin Getz16:02:29

Max 4 people per group, video cameras on, DM me to reserve your spot and I'll email you the calendar event

Dustin Getz16:02:03

Two slots left for CET onboarding tomorrow afternoon

vincent20:02:23

any slots left? also are these being uploaded anywhere perchance

Dustin Getz22:02:17

yes, i'll add you @v what's your email

dumrat10:02:20

Please add me to SGT (Singapore) this Friday Feb 24, 6pm - 8pm

👍 2
2
Dustin Getz14:02:56

We also do 1:1 onboarding zoom calls to get you up and running much faster, this will save you hours! It also helps us learn about hidden issues and figure out what docs to write, it is not a bother at all. I would personally appreciate it if you would make time to do a zoom onboarding! DM me if interested, please include your social media handle and/or a 1 sentence description of what you might want to do. (This is for my context, it is not a judgment)

😯 4
Dustin Getz15:02:32

Adam can you give more details about this question? And a link or three > @adamfrey very excited to hear that. I'm interested if it could be combined with #CVB8K7V50’s model for agnostic platform UI rendering

adamfrey13:02:39

You or the author of the Membrane library would be more equipped than I am to determine whether there's a fit between the two. But here's some context From the README: > Membrane provides all the tools you need to build a user interface. While all the pieces are built to work together, each piece is optional or can be mixed and matched with other implementations. For example, you could use https://github.com/phronmophobic/membrane-re-frame-example (re-frame) and the other layers to reach another platform. Alternatively, you could provide your own ncurses graphics backend and leverage the ui framework and graphics model. > The three main pieces are: > 1. A UI framework, membrane.component, that provides state management for GUIs > 2. A platform agnostic model for graphics and events > 3. Multiple graphics backends that provide concrete implementations for #2 > For membrane to run on a platform, the only requirements are: > 1. Drawing implementations for graphical primitives like shapes, text, and images > 2. An event loop that forwards events (eg. mouse clicks, key presses) to membrane and repaints The library allows you to describe your UI as a tree datastructure with all relevant rendering info in-band (instead of using HTML and CSS) and then there are different backends for rendering on the desktop via Skia, in the browser via webgl, browser to DOM, terminal apps, and an even experimental renderer for iOS • https://github.com/phronmophobic/membranehttps://github.com/phronmophobic/membrane/blob/master/docs/tutorial.mdhttps://blog.phronemophobic.com/what-is-a-user-interface.htmlhttps://blog.phronemophobic.com/ui-model.htmlhttps://blog.phronemophobic.com/reusable-ui-components.htmlhttps://github.com/phronmophobic/membrane/blob/master/src/membrane/example/todo.cljc ◦ this example uses all the membrane pieces together, but the library was designed so that you can just use the parts you want to

👀 2
vincent17:02:34

Hi, hyperfiddle looks like the right tool for my next project ^.^

🙂 2
👋 2
vincent21:02:14

I just cloned the repo and wonder how I should run it. Notice a shadow-cljs so probably run that... Any pointers on how to start this puppy up?

Dustin Getz21:02:25

try readme 🙂

vincent21:02:25

Ah ok I see the link to the starter app now ^^

Dustin Getz21:02:36

under Getting Started

Dustin Getz21:02:42

clj -A:dev -X user/main

Dustin Getz21:02:46

we will have to highlight it better

vincent21:02:04

hot diggity it runs 😄

😁 2
vincent22:02:06

So am I correct in assuming that the todo_list.cljc is the server file and also happens to host all the client code x)

Dustin Getz00:02:23

.cljc is a special Clojure file extension (compare to .clj .cljs) which is processed by both Clojure and ClojureScript. Clojure and ClojureScript are really separate languages with separate (overlapping) toolchains

Dustin Getz00:02:49

This all has nothing to do with Electric, this is just how Clojure and ClojureScript code sharing works through this .cljc file type

Dustin Getz00:02:56

Electric piggie backs on top of this, but in the end there are two Electric builds - a Server build and a Client build. These two builds both use the same .cljc src as inputs, but the outputs are java .class files and .js files

Dustin Getz00:02:49

The above kind of simplifies and "white lies" a bit to hopefully help your understanding, it is not strictly accurate but close enough

vincent00:02:47

yeah i like it, having both client and server in one file is cool, i appreciate the explanation

vincent00:02:38

so i am busy brainstorming new ideas now that hyperfiddle greatly simplifies realtime webapp development

🙂 2
Dustin Getz16:02:38

Group zoom onboardings this week: • PT (San Fransisco) this Thurs Feb 23, 3pm - 5pm • CET (Paris) this Weds Feb 22, 3pm - 5pm • SGT (Singapore) this Friday Feb 24, 6pm - 8pm Remember, we do 1:1 onboardings as well which are more effective, please DM me for a 1:1! On our side, the 1:1s help us figure out how to explain this better and discover hidden problems, so we appreciate if you are able to make time for a 1:1. (You can also do both.) @dumrat @kjw0323 @zenflowapp @chromalchemy @v @damir.bijuklic @mattias @sfyire @tomisme @jmayaalv @sr @jaapmaaskant

👍 2
Dustin Getz16:02:03

Two slots left for CET onboarding tomorrow afternoon