Fork me on GitHub
#hyperfiddle
<
2023-06-04
>
tobias11:06:23

Is it a bad idea to use e/for for side effects? It seems handy as a way to send parallel requests to the server. I have an app where I want to send a bunch or parallel requests to the server to do some slow operation (the server hits a third party api) and then get the results back and store them on a client-side db. Conceptually I have something like this:

(ui/button
    (e/fn []
      (e/for [thing things]
        (let [result (e/server (e/offload #(slow-calculation thing)))]
          (swap! !client-db conj result))))
    (dom/text "Submit"))

👀 1
1
tobias11:06:56

It seems to work just fine, but I don't know whether I'm making a footgun for myself 😅

braai engineer11:06:52

@UA9399DFZ e/offload’ed tasks are queued on different thread and get cancelled if button goes away, which you probably don’t want. Fine for reads, but not for writes. So you probably want to trigger one Clojure function on button click and do what you need, potentially via parallel map, not e/offload for side-effects. Use (doall (pmap …) or (doseq …), but note doseq is sequential tho, not parallel.

Dustin Getz12:06:12

your question is essentially, how to make a set of slow queries in parallel? you don’t need to use e/for for this, as Petrus says you can use e/offload to move slow things onto threads. So (do a b c) or (vector a b c) run a b c concurrently and use e/offload as needed

Dustin Getz12:06:08

Also as petrus said, the button is going to kill the callback as soon as it sees a non-Pending result, which might not be an issue here but will bite you eventually (we’re working on a much better button , we consider these semantics a misstep now)

Dustin Getz12:06:20

why do you want a button here?

Dustin Getz12:06:25

you could use a button to toggle an atom and put these requests under an if statement in the view

tobias12:06:49

Thanks for your explanations! I'll give it a go with (do ...).

Dustin Getz16:06:22

My mistake you actually will need e/for (otherwise you'd need (e/apply vector ...) which doesn't exist yet and if it did it would use e/for)

Dustin Getz16:06:03

The point is there's nothing special about this

braai engineer21:06:51

@U09K620SG is there a way to guarantee that long-running side-effects don’t get cancelled in the event the UI changes? (I’m not exactly sure under what conditions the offload is cancelled)

Dustin Getz21:06:47

Can you give me a specific example

Dustin Getz21:06:15

(e/offload #(slow-calculation thing)) isn't really a side effect insomuch as it's a value that is slow to compute

Dustin Getz21:06:44

so, if the user navigates away, they aren't interested in the result anymore and you most definitely want it cancelled

braai engineer21:06:15

OK, so only e/offload’ed calls will get cancelled? If the result is long-running, should I do (e/server (e/discard (do-slow-side-effecty-thing! conn)))?

Dustin Getz21:06:06

cancellation is linked to the idea of component-will-unmount in React

Dustin Getz21:06:38

e/offload is orthogonal to it, e/offload does indeed honor the cancellation lifecycle but cancellation is not specific to e/offload

👍 2
Dustin Getz21:06:26

This does not contain the word "cancel" but it probably should – https://electric.hyperfiddle.net/user.tutorial-lifecycle!Lifecycle

Dustin Getz21:06:19

in that BlinkerComponent demo, basically there is an if that is switching back and forth. Each time it switches, it cancels the old branch and boots the new branch

tobias23:06:26

OK I will stick with e/for . I guess the reason that I can't do pmap or (apply do my-tasks) is that in each case they need function arguments, but my task (calculating something on server and copying it to client-side db) crosses the client-server boundary so can't be wrapped up into a normal clojure function.

Dustin Getz23:06:56

i’m curious what is the client side db used for?

tobias01:06:35

I've used electric to make a few simple internal apps for my business to automate various tasks (eg analytics dashboard). For all of them I found it easier to use client side db because then the state persists even if the web socket times out. These apps are super simple, don't have any auth or back end persistence. The one I'm working on now is an app for batch conversion between image file formats. I'm "uploading" the images by converting to base64 which is little hacky but seems to work just fine. I'm enjoying figuring out how to do things in electric. Takes a different way of thinking but the resulting code is more concise (and maybe also clearer?) than it would have been with reframe frontend + reitit backend.

👍 2
Dustin Getz02:06:44

ah, are you saying that you’re using client side global state specifically as a workaround for electric losing state on reconnect? and furthermore if we fixed that, you wouldn’t? do you care about durability across page refreshes?

Dustin Getz02:06:24

i’d love to see your apps if they are on github (or add me)

tobias23:06:31

I'll check with my business partner what they're happy with me sharing.

tobias23:06:56

Yes, using client state as a workaround for losing state on reconnect. I'm agnostic about whether state is client or server side for these little apps since they're single player anyway. Durability across page refreshes doesn't matter for me at this stage.

s-ol14:06:12

can anyone share a working datomic (peer) to missionary bridge? I've tried adapting https://github.com/jeans11/electric-datomic-starter/blob/main/src/user.clj but it doesn't ever swat the db atom out

1
s-ol14:06:34

(reset! !db (d/db conn))
  (future
    (try
     (let [q (d/tx-report-queue conn)]
       (loop []
         (reset! !db (:db-after (.take ^java.util.concurrent.LinkedBlockingQueue q)))
         (println "new DB")
         (recur)))
     (catch Exception e
       (println "xcd " e))))
never prints anything. conn is intialized before this is called (and only once)

Dustin Getz14:06:39

i think there’s one in #missionary

s-ol14:06:34

oh nvm... it was working but I silently broke the db write I was testing with

👍 1
teodorlu21:06:45

Interesting. Looks a lot like Haskell do-notation to me.

Dustin Getz21:06:03

right, do-notation is the monadic style, which in scala is for-notation

👍 1
Dustin Getz21:06:50

Electric Clojure can be seen as the direct style, you "just write clojure" and still get the benefits of async composition, async error propagation, process supervision etc (the Missionary value prop)

👍 1
Dustin Getz21:06:22

To be clear, Electric is a DAG abstraction which is the Arrow typeclass not the Monad typeclass, monads don't model DAGs

1
Dustin Getz21:06:35

but Arrow and Monad are pretty close

👍 1
teodorlu22:06:01

> Electric Clojure can be seen as the direct style, you "just write clojure" and still get the benefits of async composition, async error propagation, process supervision etc (the Missionary value prop) Huh, interesting. I like how succinct Electric Clojure is. At the same time, when I read that scala article, I was wondering whether the new syntax for futures was making things less explicit and more magical. Perhaps it comes down to how neatly the primitives of the language compose together?

teodorlu22:06:25

> To be clear, Electric is a DAG abstraction which is the Arrow typeclass not the Monad typeclass, monads don't model DAGs Interesting, TIL! > Just as we think of a monadic type m a as representing a 'computation delivering an a'; so we think of an arrow type a b c, (that is, the application of the parameterised type a to the two parameters b and c) as representing 'a computation with input of type b delivering a c'; arrows make the dependence on input explicit. [<https://en.wikibooks.org/wiki/Haskell/Understanding_arrows > |source>, which refers to <https://www.sciencedirect.com/science/article/pii/S0167642399000234 > |real source>]

Dustin Getz23:06:05

yeah, the CT stuff is a lot to wade through but basically arrows are exactly DAGs and they should just say that. DAGs give you concurrency, sharing and reuse,. whereas monads model imperative statements (they should just say that also)

👍 2