Fork me on GitHub
#hyperfiddle
<
2023-05-09
>
Erich Ocean01:05:19

I'm trying https://electric.hyperfiddle.net/user.demo-webview!Webview by adapting it to the electric-starter-app repo. All seven previous examples worked out-of-the-box, but this one is throwing an error:

Erich Ocean01:05:26

------ WARNING #1 - :undeclared-var --------------------------------------------
 File: /Users/erichocean/Desktop/xygroup/electric-starter-app/src/user.cljs:8:3
--------------------------------------------------------------------------------
   5 |     hyperfiddle.electric-dom2))
   6 | 
   7 | (def electric-main
   8 |   (hyperfiddle.electric/boot ; Electric macroexpansion - Clojure to signals compiler
---------^----------------------------------------------------------------------
 Use of undeclared Var app.todo-list/conn
--------------------------------------------------------------------------------
   9 |     (binding [hyperfiddle.electric-dom2/node js/document.body]
  10 |       (app.todo-list/Webview.))))
  11 | 
  12 | (defonce reactor nil)
--------------------------------------------------------------------------------

Erich Ocean01:05:07

Here's are the relevant files I've changed, along with (unchanged) deps.edn: https://gist.github.com/erichocean/8c61a27a3ac3f6b39061c7d4054e748c

2
Vincent02:05:45

looks pretty sane to me.

Vincent02:05:54

maybe change the defonce to def and see if the problem persists

noonian05:05:26

conn is only defined in clj, therefore it will only exist on the server and I believe WebView is being called from a client context. You probably need to wrap the code referencing conn in an e/server block or call WebView from an e/server block

👍 2
Erich Ocean08:05:44

I did try that and the error went away, but the content was still missing. I'm now running it from the Electric main repository, and it does work now. (Ironically, there's now an unrelated :underclar-var warning in the compilation log. This is using top-of-tree on the main branch.)

Dustin Getz10:05:04

the demos in master are unstable currently while we land a new set of changes, please do not use master

Dustin Getz10:05:03

you can clone the examples app, see our github

Erich Ocean01:05:25

Does this example only work on a newer version of Electric?

Erich Ocean08:05:55

I'm playing with styling the Java system properties test page to test out styling with CSS. I've got my CSS file set as an absolute path in index.html (I'm editing the main Electric code). When I add a class and save, TailwindCSS updates the CSS in about 30ms, but Electric is taking 4-5 seconds to compile the code whenever I edit (dom/props {:class "tailwind-css classes go-here"}) and save.

Erich Ocean08:05:31

The entire file is just 42 lines of code, so that's…concerning.

Erich Ocean08:05:52

Anything I can do to speed up Electric re-compilation while I'm editing (dom/props {:class "tailwind-css classes go-here"}) strings, ideally avoiding it altogether?

Erich Ocean08:05:21

Okay, I commented out the other examples pages and that dropped it down to about two second to refresh after I hit save. Ideally it'd be less than 100ms, like normal shadow-cljs development.

Geoffrey Gaillard08:05:03

Tailwind just grep for class names in source files. Electric is a full compiler. We are focusing on getting the semantics right. We haven’t optimized compilation speed yet. It’s on the roadmap.

Erich Ocean08:05:37

Can dom/props {:class} be set to the value of a e/watch'd atom? Then I could just update that;

Geoffrey Gaillard08:05:27

Yes this approach would work but, it might not be ergonomic. Your current approach of commenting out everything you are not working on is ok. This is what I recommend for now.

Dustin Getz10:05:49

sorry about the long compile times we don’t have incremental builds yet. it’s a priority

Dustin Getz10:05:55

if you clone the starter app to develop (rather than the examples app with many examples) you will probably be “fast enough” for a few months while we deal with

tobias10:05:13

How would I run a js/alert on a button click? I've tried this but it doesn't work:

(e/defn MyButton []
  (ui/button (e/fn [] (js/alert "hi"))
    (dom/text "Click me")))

2
tobias10:05:29

In the console I see TypeError: Illegal invocation

J10:05:14

How MyButton is called?

Dustin Getz10:05:49

try window.alert()

Dustin Getz10:05:27

(.alert js/window …)

tobias10:05:37

That works! How come that works but js/alert doesn't?

Geoffrey Gaillard11:05:13

Looks like the electric compiler fails to resolve js/alert . I’ll log a ticket and we will give it a look. Thank you for the report.

🙏 2
braai engineer11:05:13

Does Electric handle nested signal functions e.g. tree selectors? I ported my tree picker from Rum which looks roughly like this translated:

(e/defn SimpleTreeNode [{id :xt/id, :account/keys [title children]}]
  (e/client
    (dom/div
      (dom/text title)
      (if (seq children)
        (e/for-by :xt/id [child children]
          (SimpleTreeNode. child))))))
But when I call the root node with: (SimpleTreeNode. {:xt/id :root, :account/name "Root" :account/children […]}) it throws a StackOverflowException at compile time, even when passed an empty list or vector of :account/children.

4
xificurC11:05:40

IIRC self-refer doesn't work yet. Recursive calls are still active research. Replacing the self call with recur should work for now, but note the behavior is not set in stone yet

xificurC11:05:54

ah, it's nested in a for-by, so that won't work either

xificurC11:05:57

(e/def STN)
(e/defn SimpleTreeNode [..] .. (e/for-by [..] (STN. child)))
;; call
(binding [STN SimpleTreeNode] (STN. ...))

braai engineer11:05:59

I searched for “nested” in channel I see that nested tree signals are not supported, which means I need to render this in a macro. Any gotchas? Tips & tricks? Yeah if I try to use a normal for it complains that I can’t recur there.

xificurC11:05:50

this binding hack is what we use for now

braai engineer12:05:23

Thanks @U09FL65DK! Is this solvable in future with compiler?

Dustin Getz14:05:59

"nested tree signals" what is this

braai engineer14:05:07

My made up term for recursion of signal fn’s, e.g to render a tree

Dustin Getz14:05:35

it is supported in this manner: https://github.com/hyperfiddle/electric/blob/master/src-docs/user/electric/electric_recursion.cljc - this is what Peter called the "binding hack"

grav13:05:22

I've adapted the electric-starter-app to PostgreSQL. The way I've tackled the reactivity is to add an atom with a counter (`(defonce !db-counter (atom 0))`) which gets incremented whenever I mutate the db. Then I can watch that atom (`(binding [db (e/watch !db-counter)] ...`) and pass it into all functions that need to re-run. Granted, there's a foot gun in that I need to remember to increment the counter when mutating. Also, the db argument aren't really used by all the functions that consume it. Any thoughts on a nicer solution? https://github.com/grav/electric-starter-app/blob/grav/src/app/todo_list2.cljc

grav13:05:59

Btw, I noticed that the original implementation re-queries the db whenever it wants to render an item - is that necessary? https://github.com/hyperfiddle/electric-starter-app/blob/main/src/app/todo_list.cljc#L13

xificurC13:05:36

datomic caches stuff in memory so these queries should be cheap. That's one of the value propositions of datomic

👍 2
grav13:05:22

> we have an initial guide which uses the same trick you did Ha, didn't see that one. I guess it was a good exercise to figure that out myself ;-)

Dustin Getz14:05:31

also see discussion here https://clojurians.slack.com/archives/C7Q9GSHFV/p1682786697009099 and in some other places if you search

👍 2
Dustin Getz14:05:58

Probably the long term solution is to integrate the router to refresh queries on navigate, and also refresh queries on a configurable timer (say 60 seconds), and force opt-in to get realtime refresh on a custom event for specific realtime regions of the app. The custom event could be on every pg_notify, or hopefully for realtime chat views we are using a document database designed to support realtime collection subscriptions (i.e. not sql)

chromalchemy15:05:15

I sucessfully walked through a http://fly.io deploy of Electric starter app!! 🎉 I’m a devops noob so I was a bit uncertain and trying to being extra carefull, but thankfully it was not too hard! Some notes and deploy questions in thread..

🙂 3
chromalchemy15:05:40

I didn’t have docker installed, and installed docker from Brew (on osx). This tool was limited and insufficient to build container by defualt, and I needed to install Docker desktop. Maybe other clojure project could work here too. https://github.com/lispyclouds/contajners https://github.com/juxt/pack.alpha

chromalchemy15:05:12

When I rean the flyctl deploy, it failed the first time. Citing too many restarts. I ran it again and it succeeded.

chromalchemy15:05:39

I was able to use the github secrets config for the CI action. What is an easy/simple best practice for maintaining a “secret” (api token) from local dev to production? I am looking into using https://github.com/juxt/aero#hide-passwords-in-local-private-files , but am not sure if that might be overcomplecating things.

chromalchemy16:05:57

vs: For sensitive information, such as credentials or passwords, use the https://fly.io/docs/reference/secrets.

braai engineer15:05:37

Has anybody done any work on macros to port/support Hiccup-based Rum & Reagent components to Electric? It seems that a macro that rewrites [:div.my-class [:h2 cnt “ results”]] to (dom/div (dom/props {:class “my-class”}) (dom/h2 (dom/text cnt “ results))) would go a long way. I know there is Reagent interop.

💯 2
denik17:05:22

> Since a lot of people have written a considerable amount of code in electric’s current syntax, maybe it would be worthwhile to consider a code-rewriting tool in the future? Could also be a community effort. The highest friction I’ve experienced with electric as an early adopter has been rewriting between electric-domx versions. https://github.com/hyperfiddle/electric/issues/32#issuecomment-1478393433

chromalchemy19:05:18

If a want a “ticker” message to update in the dom while a function runs.. eg showing progress of a sequence of api calls.. How would I implement that?

#?(:clj (defonce !query-status) (atom ""))
#?(:clj (defn run-query! []
          (reset! !query-status "update message 1)
          ;; do stuff
          (reset! !query-status "update message 2)))      


(e/def query-status (e/server (e/watch !query-status))

(e/defn orders-view []
   (e/client
  
  (ui/button
      (e/fn []
        (e/server (run-query!))
    (dom/text "Run query"))

  (dom/div
      (dom/text "Status: "
        query-status))))
It seems the default behavior is that the dom node does not update (from query-status : “reactive signal derived from atom”), until the server function run-query! has completely finished

Dustin Getz20:05:51

What you wrote seems fine, the dom should update when the atom swaps not be delayed until the end

Dustin Getz20:05:12

oh, the run-query fn is blocking here

Dustin Getz20:05:27

ui/button is also not the right abstraction for this, as the ui/button is expecting a single result to become available at which point it will "remove the callback node from the DAG"

Dustin Getz20:05:39

Can you wait a week or two on this? We have a bunch of new UI controls coming that will help us all think more clearly about problems like this

Dustin Getz20:05:40

If you do make the query async like (e/offload #(run-query!)) it will probably fix your first problem but then i would expect it to hit the second problem and kill the query half-way completed

chromalchemy21:05:56

Ok thanks for context. It’s not critical behavior for me atm. Looking forward to the evolutions!!

chromalchemy21:05:38

I was able to work up a functioning dashboard up this afternoon 🎉!! (with operating server code I already had). Beautiful to see it working! I like that I can update the code from both sides and have view reloaded. Loving the github CI! Now I don’t have to think (worry) about deployments steps so much already.

🙂 2
chromalchemy22:05:46

Electric gave me confidence to go ahead and deploy a whole APP for a limited use case where our SAS api was just not going to give us a simple query filter we needed.

2
👏 1
chromalchemy19:05:53

I am getting an error with my longer query function. It is basically a do-seq that is going through 1400 orders, doing an api request query on each one, then when it matches a predicate, appending the query results to a server side atom (which is synced to cliend with e/def ) > i would expect it to hit the second problem and kill the query half-way completed Is this what you were referring to?

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] ERROR hyperfiddle.electric-jetty-adapter: Websocket handler failure

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] clojure.lang.ExceptionInfo: Websocket pong timeout.

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at hyperfiddle.electric_jetty_adapter$make_heartbeat$cr13526_block_4__13536.invoke(electric_jetty_adapter.clj:22)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at cloroutine.impl$coroutine$fn__10330.invoke(impl.cljc:60)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.Sequential.step(Sequential.java:86)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.Sequential$1.invoke(Sequential.java:109)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.core$absolve$fn__10694$fn__10695.invoke(core.cljc:141)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.RaceJoin.terminated(RaceJoin.java:51)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.RaceJoin$1.invoke(RaceJoin.java:71)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.RaceJoin$2.invoke(RaceJoin.java:86)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.Sleep.trigger(Sleep.java:63)

2023-05-11T19:36:56.987 app[3d8d0d2f296589] mia [info] at missionary.impl.Sleep$Scheduler.run(Sleep.java:35)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] ERROR hyperfiddle.electric-jetty-adapter: Websocket error

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] java.lang.NullPointerException: null

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at hyperfiddle.electric_jetty_adapter$electric_ws_adapter$on_text__13579.invoke(electric_jetty_adapter.clj:76)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at ring.adapter.jetty9.websocket$proxy_ws_adapter$fn__13428.invoke(websocket.clj:155)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at ring.adapter.jetty9.websocket.proxy$org.eclipse.jetty.websocket.api.WebSocketAdapter$WebSocketPingPongListener$12d400b6.onWebSocketText(Unknown Source)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:296)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:67)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:235)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:152)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:326)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:148)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:111)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:169)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:90)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:459)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:440)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:383)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)

2023-05-11T19:36:56.988 app[3d8d0d2f296589] mia [info] at java.base/java.lang.Thread.run(Thread.java:829)

👀 1
Dustin Getz19:05:31

That looks like you blocked the server reactor with a blocking clojure function

Dustin Getz19:05:12

Try (e/offload #(run-query ...)) to move it to a thread pool

Dustin Getz19:05:33

Be aware that e/offload has a subtle issue that causes over-rendering that we haven't fixed - you probably won't notice but I wanted to note it

chromalchemy20:05:07

Ok, will try that. Thanks

chromalchemy21:05:54

It worked! Server fn completed with only adding that e/offload to call. I hear you on the subtleties and works in progress. This was a minimal mvp for me. I still have yet to explore the api in more depth. But it was great to see I could do this with minimal syntax. Nice to see table updating too. Thanks so much for the quick advice.

🙂 1
Dustin Getz21:05:02

Can I see your final code?

chromalchemy21:05:14

here is the repo

chromalchemy21:05:33

It’s probably bit overcomplicated for what it’s doing. I thought at first I had to declare the atoms in cljc file, so I factored the clj file functions to be pure and take the atoms as args. But would probably keep them global in in a re-write. Or just break the clj code down so it could be called more atomically from electric and only use e/def, and append to it on the client.

Erich Ocean22:05:48

Are there docs anywhere that describe the compilation model? For instance, are e/defn's inlined into their caller somehow, when graph coloring is done? Or is it purely lexical? If I use an e/defn from another namespace, will I have any problems related to separate compilation using it? The problem I'm trying to solve here is to have a library of Electric components that can be separately developed (and especially, styled since that workflow is so slow currently) and then use them to build up apps?

1
Erich Ocean22:05:35

The UIs I'm working on are at least an order of magnitude more complex than the demos.

Erich Ocean22:05:51

I'm willing to put up with difficulties while it's in alpha because I really like the overall approach, especially when paired with Datomic on the back end (which is what we use).

Erich Ocean23:05:02

For example, in the Electric repo, when I comment out all of the pages except the one I'm working on in user_main.cljs, the compile time drops a lot. So clearly, ALL pages/namespaces are being re-compiled when one page is updated—even though they haven't changed. Is there something that requires every e/defn function to be re-compiled in order for Electric to "work", or is this just a missing compilation optimization opportunity?

Erich Ocean23:05:32

i.e. during compilation, we could (in principle) memoize the result for each e/defn we're compiling, and if it hasn't changed, would the previous compilation result still be valid?

Dustin Getz23:05:02

Function by function compilation is not yet possible on the current implementation. We are working on it. The current Electric network planner is implemented as a whole-program optimization pass. Some of the network optimizations are done in this compile-time analysis and the optimizations are non-local, i.e. an efficient network plan cannot be inferred without knowledge of the entire program. That's why, today, the whole Electric program (across all namespaces) is compiled all at once for every change. We are already deep into major Electric internals improvements which will unblock many quality of life improvements like incremental builds and function-by-function builds. We are working towards moving certain network optimizations to runtime (i.e. a JIT compiler, like the JVM). That will enable incremental builds. That's great that you want to build complex UIs with Electric. We, too, are building complex things with Electric.

Dustin Getz23:05:01

Also, it seems like you are still running demos from Electric master? Please do not do that. Master is not stable right now due to landing the aforementioned changes. There are also lots of undocumented changes, including to the demos. Please fork the starter app https://github.com/hyperfiddle/electric-starter-app

Erich Ocean23:05:27

I did fork that, but one of the demos didn't work so I switched to master to get it working.

Dustin Getz01:05:38

what exactly is broken? we will fix it

Erich Ocean23:05:37

I'll switch back.

Erich Ocean23:05:51

But back to the original question, thanks for the detailed answer.