Wrote down some thoughts on our internal subscription layer this morning and thought maybe it’s interesting to share here as well. It’s more questions than answers really but I’m curious if it prompts any reactions/thoughts from other folks working on this type of stuff.

Probably a third approach worth mentioning is using a database type thing on the client that allows for efficient queries combining data (e.g. entitydb or autonormal) but I think that still leaves a few things open, in particular around the time stuff.

makes sense to me

the watch vs. delta vs. database thing I think you're right you can kind of boil all of them down to events

one thing I've been thinking about for autonormal is adding delta capabilities to it. since it knows all of the entities you're adding to the map in order to normalize them, I think you can reliably obtain a "which entities changed" delta. so instead of listening to the whole db, you could write a wrapper that listens just to the entities you care about

the bones of this are already in there with add-report and pull-report it just needs to be exercised to see if if it's reliable

ofc if your data source already has these delta events then that might be more efficient, assuming the events are at the level of granularity you want

w.r.t. time I think your idea of setting a timer is best. you can have an object that tracks end_dt and clears and recreates the timer when end_dt changes

ensuring accuracy of the timer is a little tricky too

I could imagine something like:

(defn use-timer
  [end-dt on-end]
  (let [cb (useRef on-end)]
    ;; ensure ref is updated after each render so we always
    ;; have the latest on-end fn
    (useEffect #(set! (.-current cb) on-end))
    (useEffect
      (fn []
        (let [interval (js/setInterval
                         #(when (>= (js/Date.now) end-dt)
                            ((.-current cb)))
                         ;; check every 100ms
                         100)]
          #(js/clearInterval interval))
      #js [end-dt])))

Yeah that’s an approach we use in a few places but I’d almost like to handle it outside of react closer to the layer that transforms data for usage in views

(which is totally possible ofc)

@lilactown with autonormal, I was wondering: is there a functionality to identify records with “slots” like in entitydb or idx?

not sure what you mean

I tried ctrl-f "slot" in entitydb and idx's READMEs

ah I see the previous iteration of entitydb has it...

ah so like a custom index. yes I think you can do that with autonormal

(-> (a/db [])
    (a/add {:users {:current {:user/id 1 :user/name "martinklepsch" :user/likes #{"programming"}}}}))
;; => {:users {:current [:user/id 1]}
;;     :user/id {1 {:user/id 1 :user/name "martinklepsch" :user/likes #{"programming"}}}

e.g.

(a/pull *1 [{:users [{:current [:user/name]}]}])
;; => {:users {:current {:user/name "martinklepsch"}}}

is that what you mean?

> since it knows all of the entities you're adding to the map in order to normalize them, I think you can reliably obtain a "which entities changed" delta. so instead of listening to the whole db, you could write a wrapper that listens just to the entities you care about I think this would be useful for @roman01la’s thoughts around "kafka for the frontend" too. I've personally run into re-frame performance issues where I really just needed a delta & to update indexes myself rather than rebuilding them from scratch.

Re-frame’s inability to reason about time does make it very cumbersome to do stuff like that, yeah

(In subscriptions)