Fork me on GitHub
#clerk
<
2023-01-24
>
Matthew Downey17:01:04

Hey, I'm running into some behavior with syncing atom state that I don't quite understand, which seems to manifest itself when recomputing the namespace becomes more costly. Putting details in a thread to keep the channel clean, but I'd be appreciative if anyone knows what might be happening!

Matthew Downey17:01:15

So I have a notebook with a number input control more or less copied from https://github.com/nextjournal/clerk-demo/blob/main/notebooks/controls.clj in nextjournal/clerk-demo. My minimal reproduction is 88 lines (https://gist.github.com/matthewdowney/be1c88c2d3d72312bd2bae5bf7788809).

Matthew Downey17:01:33

With a trivial example, the control works fine, just like in the demo. I'm plotting some number of simulated random walks using the control value as an input. When I change the input's number in the browser, the plot updates as expected. However, when I increase the number of random walks that I'm plotting from 10 to 100, the control in the browser stops working. State does not update in the browser or on the back end (though I see "Clerk recomputed..." messages). Weirdly, if I just load the whole namespace into the REPL, it loads pretty quickly, so it's not like it's the time that it takes to evaluate the code that is causing this behavior. Sometimes, it will eventually update even with the higher workload, but it seems like some part of the sync is not working as I was expecting.

Matthew Downey17:01:38

I'm curious if anyone has either (1) run into something similar and solved it, or (2) can just explain more about how the sync / reload process works so that I can debug this.

mkvlr14:01:02

hi @UP7RM6935, thanks for the repro, taking a look at this now

Matthew Downey14:01:30

Sure thing, thanks for checking it out!

mkvlr15:01:33

ok, so the problem is that the diff computation is really slow in this case

mkvlr15:01:32

editscript is trying to compute a minimal change operation for all the points in the plot

Matthew Downey15:01:47

Ohhh that makes a lot of sense

mkvlr15:01:50

think the solution is to disable that for results like plotly

mkvlr15:01:37

also here’s a bit simplified version of your script:

; # Syncing atom state + long computations
(ns sync
  {:nextjournal.clerk/visibility {:code :hide :result :hide}}
  (:require [nextjournal.clerk :as clerk]
            [nextjournal.clerk.viewer :as viewer])
  (:import (java.util Random)))


(def integer-input
  (assoc viewer/viewer-eval-viewer
         :render-fn
         '(fn [!state]
            [:input
             {:type     :number
              :default-value    @!state
              :class    "px-3 py-3 placeholder-blueGray-300 text-blueGray-600 relative bg-white bg-white rounded text-sm border border-blueGray-300 outline-none focus:outline-none focus:ring w-full"
              :on-input #(reset! !state (js/parseInt (.. % -target -value)))}])))

{:nextjournal.clerk/visibility {:code :fold :result :show}}

                                        ; For each $1 you bet, you win either $X or $0, with 50% probability, where X =
^{::clerk/viewer integer-input ::clerk/sync true}
(defonce payoff (atom 2))

#_(reset! payoff 2)

                                        ; A simulation of many portfolios making this bet a hundred times with 10%
                                        ; of their bankroll:

^{:nextjournal.clerk/visibility {:code :hide :result :hide}}
(defn portfolio
  ([bankroll ^Random r]
   (portfolio bankroll r 50 @payoff))
  ([bankroll ^Random r winprob winpayoff]
   (lazy-seq
    (let [bet-size (max (* bankroll 0.10) 0)
          bet-result (if (< (.nextDouble r) (/ winprob 100.0))
                       (* winpayoff bet-size)
                       0)]
      (cons bankroll
            (portfolio
             (-> bankroll (- bet-size) (+ bet-result))
             r winprob winpayoff))))))

(defn medians [data]
  (let [data (vec data)]
    (apply mapv
           (fn [& values-at-time]
             (let [sorted (vec (sort values-at-time))]
               (nth sorted (quot (count sorted) 2))))
           (map :y data))))

(defn with-medians [data ms]
  (conj
   (mapv #(assoc % :showlegend false :opacity 0.25) data)
   {:y ms
    :showlegend true
    :type "scatter"
    :line {:width 3 :color "firebrick"}
    :name "Median"}))

(def n-portfolios 100)

^{::clerk/width :full ::clerk/viewer clerk/plotly}
(def plot
  (time (let [data (for [i (range n-portfolios)]
                     {:y    (into [] (take 100) (portfolio 100 (Random. i)))
                      :opacity 0.15
                      :type "scatter"})
              counting (completing (fn ([] 0) ([n _] (inc n))))
              n-lost (transduce
                      (comp
                       (map (comp peek :y))
                       (filter #(< % 100)))
                      counting
                      data)
              ms (medians data)]
          {:data (with-medians data ms)
           :layout {:title (str "Betting 10%"
                                ", median final bankroll " (biginteger (peek ms))
                                ", " n-lost " portfolios lost money")
                    :xaxis {:title "Bet #"}
                    :yaxis {:title "Bankroll" :type :log}}})))

mkvlr15:01:57

need to run now, can do further optimizations if needed tomorrow, let me know

Matthew Downey22:01:00

Oh hey thank you, huge improvement as of that latest commit (and thanks for that nicer way of defining the input!). Still somewhat slow to update compared to the evaluation time locally, but I think I also understand better what is being syncd with the browser and when I might want to prefer evaluating things from the CLJS vs CLJ side.