we just serialize our app-db to EDN on the filesystem when the app is backgrounded, and retrieve from EDN when the app has been evicted and is started or foregrounded

Oh, that’s a good idea. I hadn’t thought of leveraging backgrounding/foregroudning.

What does “has been evicted” mean? (I’m relatively new to app development)

It’s a little scary though, because if the app crashes before it’s backgrounded, the user would lose all of their work.

"evicted" means that, after backgrounding, the app's paused process image was killed by the o/s, so when next foregrounded it's starting from scratch

in the case of our app we also have most state server-side, so we are only storing locally so that the app can continue to work offline - so the risk of the user losing anything is small

if i needed to store everything then i would consider storing an occasional app-db snapshot, and then appending in a separate file, a list of mutation events to be applied to the app-db snapshot

you can probably arrange that neatly with a re-frame interceptor

and since you would only be appending event data (which will serialize nicely to EDN, since your re-frame events / app-db are all pure, simple data) to an already open file it will be blazingly fast

that’s actually the exact strategy I was using previously

i ran into a couple of issues with that. 1) Not everything in the event data was neatly serializable (e.g. some cofx data had infinite sets or fns etc) and 2) It’s possible that the implementation of the event handlers could change between the time the event was stored and when it was played back

I think 2) probably isn’t much of an issue 99.9% of the time, though

[2] you can deal with in a similar way to how you would deal with e.g. messages from kafka being processed after a db schema changes - either make sure schema changes are backwards compatible (only additions - no field deletes / type changes until you are sure everything has been flushed) or drop changes with stale schema

[1] are you putting things which are not pure, simple data into your app-db ? if so, try and avoid doing that. it causes pain 😬- could you store a pure-data description of a seed or generator for the infinite set instead ?

[2] Yeah that’s a good call. I had thought about doing that as well. Maybe I’ll open source my current implementation and release it as a re-frame plugin/interceptor.

[1] Not in the app-db, but in the cofx and args to the event message itself.

ah, i see what you mean for [1] ... in which case, you could perhaps look at the :db fx itself, after the event has processed, and use clojure.data/diff to extract a minimal diff to write to your change-log - cf https://clojuredocs.org/clojure.data/diff

i've only used the clojure.data/diff stuff for single-level diffs myself - but the docs indicate it's ok for nested structure diffs

I’m actually doing just that haha. Using a lib called differ https://github.com/Skinney/differ

Good to know that at least one other person is having the same thought process as me

😅

Started out serializing and storing the entire db. Then moved on to just storing a list of cofxs and replaying them. Then after my concerns of [1] and [2], I’m using the diffing strategy you described.

and that’s where I am now

👍