Fork me on GitHub
#hyperfiddle
<
2023-09-04
>
Can10:09:40

Hello everyone, I have a question about how advanced routing system works in Clojure / Electric. Can you take a look and if possible help me please? https://stackoverflow.com/questions/77036370/how-parametered-routing-system-works-in-clojure-hyperfiddle-electric-dsl

telekid13:09:30

I haven't anything close to a canonical routing story for electric apps (nor would there necessarily be?) Have you come across https://github.com/hyperfiddle/electric/blob/master/src/contrib/electric_goog_history.cljc and https://github.com/hyperfiddle/electric/blob/master/src/contrib/sexpr_router.cljc? I believe the latter is used to store app state in the URL for https://electric.hyperfiddle.net/.

telekid13:09:07

I'm prototyping an electric-focused router called https://github.com/telekid/pondo, but I wouldn't use it yet (and there are no docs)

👀 2
Dustin Getz13:09:38

@U04SVJW7DLZ the hyperfiddle.router ns is experimental, we make breaking changes, i don't recommend it for beginners you have to read the source

Dustin Getz13:09:40

You can also try using core.match for the router now that we support core.match

Dustin Getz13:09:09

in combination with contrib.electric-goog-history (https://github.com/hyperfiddle/electric/blob/master/src/contrib/electric_goog_history.cljc), which is very simple, 20 LOC, probably the minimum viable history integration

Dustin Getz13:09:39

contrib namespace also may have breaking changes in the future, sorry. But you can always copy/paste this and maintain it yourself as it is 20 LOC

Kein15:09:31

I have a function:

(e/defn create-data! [data]
 (e/server (d/transact! !conn [ (Transform (Query. data)] ) ;The Query. is an e/fn 
) 
With its context:
(e/defn Home []
  (Event-System.)
)

(e/defn Event-System []
  (let [event (e/watch !event)]
    (when (= (:status event) true)
      (Dispatch-Event. (:event event))
      (reset! !event event-default))))

(e/defn Dispatch-Event [event]
  (let [event-map  {:create-data create-data!}]
    (new ((:type event) event-map)  (:edata event))
    )
  )
The create-date is supposed to be reactively computed when I press “Enter” , but instead there is no effects, i.e. the transaction did not execute. [0]

Kein15:09:00

What confuses me is: [1] When I change the code to the following, and presss “Enter”, it worked , showing “ASD” in JS console. This means that the code could be executed.

(e/defn create-data! [data]
 (println "ASD")
) 
[2] In the REPL I tested the following, successfully mutate the state, and get the correct result showing in the UI.
(d/transact! !conn [ (Transform (Query data)] )  ;the Query is a non-e/fn normal query
[3] When I hook the transaction in an e/fn inside an e/button (as the following), the transaction ran, but 2 items in my todo-list disappeared, and when I refresh the browser , no data can be queried.
(e/defn Send [] (ui/button (e/fn []
                             (create-data!. {:uid "test7" :value "test13"})
                             )
                           (dom/text "Send")))
[3.1] Both Datalevin and Electric have no runtime error. [4] After [3], the “/tmp/datalevin/mydb” is broken. When I query and transact, there is no response. [5] After [4], I reload the page 2-3 times and hit the ‘transact’ button 2-3 times , the memory usage increased from 1GB to 4GB and crashed the system.

Kein15:09:14

I’m not sure whether there is some un-idiomatic process, and I’m not sure why [1]-[5] happens.

xificurC15:09:14

what is Transform?

xificurC15:09:18

I don't understand the purpose of Event-System but it sets my spidey senses tingling. Electric is a continuous time language whereas events are discrete. If you expect !event to act as a sort of event stream note that you will hit a case where electric skips some of them

xificurC15:09:13

maybe your ultimate goal can be accomplished by other means. What problem are you trying to solve?

Kein02:09:07

The create data receives {:target-id x :value y } , and Transform turn this data into a datalog transaction body : [ {:db/id -1 , :value y } { :db/id (Get-ID x)} :next -1 ]

Kein02:09:05

The purpose of the event system is to do e/fn inside reagent component. I find no way to hook e/fn inside the event-listener in reagent component. So I have to cache the event in a global atom. Is this an idiomatic way? Maybe there’s another approach?

Kein02:09:51

In Dispatch-Event, the event-map is something that we register new function there.

xificurC06:09:19

is Transform a clojure or electric function?

Kein06:09:41

A clojure function , written locally inside create-data!

xificurC07:09:59

regarding reagent components, we haven't settled on a singular solution but hepled users on a case by case basis. If the events you care about are DOM events the simplest approach is

(def !my-div (atom nil))
... somewhere in reagent ... (reset! !my-div this-div)
... in electric (when-some [my-div (e/watch !my-div)] (dom/on my-div "click" ..))

👀 2
xificurC07:09:14

> A clojure function , written locally inside create-data! I'm asking because we titlecase electric functions to disambiguate them from clojure functions. create-data! is an electric function, is Transform defined inside the e/server block?

Kein07:09:15

Transform is not a function, but an abbreviation of some operation inside e/server

Kein07:09:28

It’s the following:

(e/server
     (d/transact! !conn
                  [{:db/id (:db/id (Get-Node-From-ID. "test7"))
                    :next -1}
                   (merge
                    {:db/id -1,
                     :user "Kein",
                     :uid "test13",
                     :parent (:db/id (Get-Node-Relation. "test7" :parent)),
                     :value "test13"}
                    (if-let [x (:db/id (Get-Node-Relation. "test7" :next))] {:next x} {}))]))

Kein07:09:57

The Get-Node-From-ID and Get-Node-Relation is abbreviated as Query.

Kein07:09:44

I think the primary difference between our approaches is that, I use a normal watch and a normal function Create-Data! , and you hook an extra e/fn in dom/on. Are there any difference between the 2?

xificurC07:09:34

I think I know what the problem is. > I use a normal watch and a normal function Create-Data! , and you hook an extra e/fn in dom/on The reason I want to use dom/on is it has known semantics, e.g. I can tell what happens if a new event arrives while the previous one is still executing on the server. The same can be said about the ui4 controls, which are not final but have been ironed out to have reasonable semantics at this point. So e.g. a ui4/button is a transcactional button, it's disabled while the server is processing a request. Generally speaking electric is a concurrent language and building these little machines can be tricky. Looking at your event system implementation

(e/defn Event-System []
  (let [event (e/watch !event)]
    (when (= (:status event) true)
      (Dispatch-Event. (:event event))
      (reset! !event event-default))))
Dispatch-Event and reset! run concurrently. What happens in your case is • Dispatch-Event runs create-data!, which has a server block, so it throws Pending • concurrently reset! changes !event, the watch fires • now I guess when returns false, unmounting Dispatch-Event • therefore create-data! unmounts as well, before the server had the chance to finish I'd advise to use dom/on and the ui4 namespace where possible. If you're interfacing with a reagent component and can't use ui4 and dom/on doesn't have the correct semantics please describe the semantics you wish to achieve and I can help building it

👀 2
Kein14:09:54

@U09FL65DK Regarding:

(def !my-div (atom nil))
... somewhere in reagent ... (reset! !my-div this-div)
... in electric (when-some [my-div (e/watch !my-div)] (dom/on my-div "click" ..))
[Q1] What can this-div be, any level of reagent-component ? Such as:
[:> Slate
        {:editor editor
         :initialValue [{:children
                         [{:text (:value data)}],
                         :type "paragraph"}]
         :onChange (fn [value])}
        [:> Editable ]
] 

Kein14:09:59

[Q2] And should we do a lot of (dom/on) for different events or even different kinds of div? Do you have a file containing an implementation of this pattern (or more context) ?

Kein14:09:49

[Q3]

(e/defn Event-System []
  (let [event (e/watch !event)]
    (when (= (:status event) true)
      (Dispatch-Event. (:event event))
      (reset! !event event-default))))
When !event change, which functions run, and which runs first? [A] when [B] Dispatch-Event They all have event which is watched.

xificurC14:09:00

Q2 was discussed several times in the channel, here's probably https://clojurians.slack.com/archives/C7Q9GSHFV/p1683292217371419. I'd do it as if it was in electric, per DOM node

Kein14:09:51

OK Thanks

xificurC14:09:24

Q1 I'm not sure what Slate returns, it's a react wrapper that probably builds a lot of DOM nodes. You'll have to poke around. Options I see • the https://docs.slatejs.org/libraries/slate-react/event-handling are normal DOM event handlers installed on a top-level DOM node, in which case you can use the trick described above • the events/callbacks you want to handle are not plain DOM event handlers. Here it gets trickier and you'll need to bridge to electric differently, probably through m/observe and using one of the event handlers from electric such as e/for-event • there's a non-react slate package, maybe it'd be easier to create a native electric wrapper?

👀 2
xificurC14:09:42

if you end up with the second option you could create a repo I can clone and start easily and show me where you got stuck and I can flesh out the initial implementation for the bridge

xificurC14:09:55

Q3 I think when will run first, but I'm not sure. You can add some printlns to double check

Kein14:09:43

[Option1 (O1) -Q1] Peter, what do you mean by normal DOM event handlers and not-plain DOM event handlers? [O1-Q2] The trick is the following, right? [Q2.1] So you mean that we do the reset! in the normal event handler? [Q2.2]

(def !my-div (atom nil))
... somewhere in reagent ... (reset! !my-div this-div)
... in electric (when-some [my-div (e/watch !my-div)] (dom/on my-div "click" ..))

xificurC14:09:46

https://docs.slatejs.org/libraries/slate-react/event-handling say > By default, the Editable component comes with a set of event handlers that handle typical rich-text editing behaviors (for example, it implements its own onCopy, onPaste, onDrop, and onKeyDown handlers). If these are equivalent to basically document.getElementById('editor').addEventListener('keydown', e => ...) then I mean they're "normal". If they are called onKeyDown but don't map 1:1 to the DOM keydown event then they're "not plain"

👀 2
Kein14:09:27

I see. I’ll take a look at the m/observe and others for a while. Thanks a lot 🙂

👍 2
Kein13:09:14

I tried bridging reagent with electric using for-event-pending-switch provided by Peter. But finally it turned out that the issue is from Query. . I should let the Query function be normal instead of an e/fn. It seems that if I use e/fn, the function will keep updating while I want it to be computed only once. Or are there any better explanations?

xificurC13:09:00

you should write electric functions when you require client/server transfer

xificurC13:09:24

> if I use e/fn, the function will keep updating while I want it to be computed only once that probably isn't the case, (q x) and (Q. x) will both react when x changes

Kein13:09:30

As long as (q x) is defined in an e/defn , (q x) is reactive, right?

xificurC13:09:41

what do you mean by "defined in an e/defn"?

Kein13:09:16

In an e/fn , every expression is a DAG node, which means that the return expression and all its dependent expressions are DAGs

Kein13:09:55

[Q4] So in an e/fn, if both (q x) and (Q. x) are dependencies of the return value, they are both reactive? I.e. in the following :

(e/defn Fn1 [x y] 
  (fn2 (q x) (Q. y)) 
)
[1] When x change, q is recomputed , if (q x) is different then the previous result , Fn2 is recomputed [2] When y change, Q. is recomputed , if (Q. x) is different then the previous result , Fn2 is recomputed

xificurC14:09:51

correct

🙌 1
nakkaya15:09:35

I have a map with the following structure

{coll-id {query-id-a {:query "<some query>", :response ["list" "of" "tokens"]}
          query-id-b {:query "<some query>", :response ["list" "of" "tokens"]}
          query-id-c {:query "<some query>", :response ["list" "of" "tokens"]}}}
the :response is streamed from a LLM model, each new token generated is appended to the :response vector. Then on electric side I have,
(try
  (e/server
   (e/for [[_ q] (responses coll-id)]
     (e/client
      (dom/div
       (dom/props {:class "mt-3"})
       (dom/h4
        (dom/text (:query q)))
       (dom/div
        (dom/text (apply str (:response q))))))))
  (catch Pending _))
During the update the latest response (the one getting the updates) text is flickering while older responses are staying solid. Is there a work around to avoid flickering?

xificurC15:09:22

could the cause of flickering be that the DOM nodes are unstable? If so you need to make sure e/for or e/for-by is stable across runs of responses

xificurC15:09:27

I think (e/for-by first ...) will do the trick

xificurC15:09:05

right now e/for is stabilizing on the whole [_ q] where the latest response is still changing. If the first part is stable you need to use it as the e/for-by key

nakkaya15:09:59

> If so you need to make sure e/for or e/for-by is stable across runs of responses Can you expand a little bit on what this means? In my example case values for query-id-a and query-id-b are not changing. Only the :response in query-id-c is updating. This is the place where the flicking happens. div s for the first two items stay solid as do the :query part of query-id-c . But the :response part is flickering with each redraw. (if this would be the correct terminology.)

xificurC18:09:49

Sure! e/for-by manages values for you. Each unique value is rendered separately. When your query re-runs it needs to decide which renderings need to be added, updated or deleted. This is e-for-bys first argument, a key-fn, which e/for defaults to identity. If (= previous (key-fn current)) it's an update, if not it is an add/delete. BTW this is similar to react.js key

nakkaya20:09:29

@U09FL65DK thanks for the explanation, that worked. I was wondering why the old scheme did not flicker but the new one did.

Dustin Getz21:09:47

e/for should just never be used, we will remove it

Can16:09:59

Hello, I am faced with a strange error. While I developing one page, I clicked a button in the browser then suddenly I got this error and now I getting the same error. What I did: - Restarted pc - Ofc undo - I recloned the newest electric version and implemented my code result is unfortunately the same. - I deleted DB and recreated but nothing fixed. - I deleted the last page which I working on but still getting the same error. Until I click that button everything was working nicely and without problem.

Dustin Getz16:09:23

To debug this I recommend commenting out parts of your app to try to triangulate which piece of code in your app is causing the problem

Dustin Getz16:09:42

(Thank you for upgrading electric versions, that gives us a common baseline point)

2
Dustin Getz16:09:27

Ah, you stated that it is the button that causes the problem

Dustin Getz16:09:04

Can we comment out pieces of the e/fn in the button to figure out which expression breaks it?

Can16:09:38

Hello, I solved but needed to delete 3 pages 🙂

Dustin Getz16:09:52

I think you can triangulate it better than 3 pages

2
Can16:09:02

I deleting my post thank you so much for separating time.

Can16:09:25

I deleted single by single which didn't solve the problem, I tried to delete group by to find errors scope then I found out that it works if I delete like that.

joshcho18:09:48

I have a fairly complicated defn that I don't want to turn into e/defn. I want to be able to do a blocking call with the server, where I ask for some value from the server and the execution doesn't move along until it get it. Also, I would be open to changing it to a e/defn, but sometimes reactivity makes things not so straightforward.

Dustin Getz18:09:31

can i see sample code

joshcho18:09:29

process-stack is the function I am keeping as defn

joshcho18:09:54

process-stack takes in stack (for use in Forth) and a value (which is just a string like "dup dup"), and updates the stack. The third argument realize? is there so that calling with realize? = true actually does the thing, whereas if it's false, only a preview of the result is shown.

Dustin Getz18:09:19

can you re-upload that as an attachment please, otherwise cannot view this thread in mobile

👍 2
joshcho18:09:49

does that work?

Dustin Getz18:09:07

set filetype to clojure please as well

👍 2
Dustin Getz18:09:02

what is cmt p/peek etc

joshcho18:09:12

Oh those are just logging functions

joshcho18:09:28

p/ is logging , tw is tailwind

Dustin Getz18:09:58

if you can reduce the LOC by removing noise to make this easy for me that would help, which line is the blocking line?

joshcho18:09:01

I can try removing some stuff if that will help (I sometimes end up removing too much)

Dustin Getz18:09:46

`

(enter
                (e/fn [s]
                  (when-not (re-matches #"\s*" s)
                    (process-stack stack s true))
                  (reset! !value "")))

Dustin Getz18:09:26

you said it's supposed to run on the server?

joshcho18:09:49

No, all of this right now runs on the client, but I want to extend this to be able to have some parts run in the server

joshcho18:09:00

process-stack right now is bound to client atm

joshcho18:09:06

I can't get it to work when I change it to e/defn

joshcho18:09:16

It works for enter, but doesn't work for preview

joshcho18:09:53

cleaned up Main

Dustin Getz18:09:05

generalizing, if you have a structure like (case (e/server (foo ...)) (println ::done)) the conditional node case will be forced to wait for the pending value to resolve to know which branch to run. You can exploit that in this way, like here, to run the println after the remote value has resolved

❤️ 2
Dustin Getz18:09:45

you could use if-some for example to wait for the value and then continue into the body with the result bound

joshcho19:09:15

IIUC, the problem rn is that I can't put e/server in because the process-stack function is just a defn. Ideally it would become a e/defn, or do a non-Electric thing within the defn. There's probably something wrong with my Main function logic

Dustin Getz19:09:02

you'll need to perform the e/server in an e/defn and pass in those values to the clojure fn

joshcho19:09:09

the e/defn version

Dustin Getz19:09:26

alternatively you can call the clojure function on the server and then do any clojure stuff (like block) inside that function

joshcho19:09:47

The waiting for values are not trivial, it's recursive and intertwined together.

joshcho19:09:56

That could work

joshcho19:09:37

Is there something wrong with the way I am using process-stack here? The first process-stack. in Main isn't called when value updates. (value and stack are watches).

joshcho19:09:01

I also tried something like

(new
 (e/fn [stack value]
   (process-stack. stack value false))
 stack value)
but doesn't quite work.

joshcho19:09:41

I resolved it using Missionary, but maybe isn't necessary.

(e/for-event
 [v (e/fn [] [stack value])]
 (process-stack. stack value false))

xificurC07:09:24

could you explain what the goal is? What do you wish to move to the server in process-stack?

nivekuil19:09:57

why does ui/input lose focus every time it re-renders? code for reference https://gist.github.com/nivekuil/5518e363785763f1c1bbf4d514594a0e

Dustin Getz19:09:03

our live demos demostrate that in fact this does not happen

Dustin Getz19:09:23

in your code if editing? turns false, the input will unmount, is that what is happening?

nivekuil19:09:55

still happens even I get rid of the if

Dustin Getz19:09:06

i'll need to reproduce

nivekuil19:09:23

ok, I'm just 20 minutes into hacking the starter app, file as is

nivekuil19:09:44

with net.sekao/odoyle-rules {:mvn/version "1.3.0"}

Dustin Getz19:09:51

try to repro without that

nivekuil19:09:03

without what?

Dustin Getz19:09:17

adding third party stuff

Dustin Getz19:09:32

the e/for does not have a key, is that the issue?

nivekuil19:09:00

why does it happen with only 1 element though?

Dustin Getz19:09:59

the rules engine is returning a new reference with [possibly] equal value

Dustin Getz19:09:38

we never shoulda published e/for, this is a foot gun

Dustin Getz19:09:57

it will be removed in electric v3 probably

Panel01:09:08

What should be used instead of e/for ?

henrik06:09:31

Presumably e/for-by identity, forcing you to be explicit about wanting whole value equality comparisons.

👍 4