Clojurians Log v2

It’s always been true that Fulcro is a reasonable fit for any kind of sync engine (via websockets, etc). I have a query subscription system I wrote for Datomic running in one of my production products over websockets. It’s actually a great match IMO, since you can subscribe to a particular query or set of queries, and you get auto-normalization and render without much trouble. But it is true that I’m not promoting an actual db synchronization ala meteor. Keeping an entire db up to date from the server, IMO, is a wonderful idea, and terrible in practice. I have production customers with hundreds of thousands of entities. I don’t want to sync that mess and keep it up to date. WAY too heavy, and the resulting distributed systems issues are quite complicated (what if I have two active customers hitting the db in similar locations at the same time? Sure, you can argue that a db sync system could support some kind of distributed transactional support, but then you actually have to make that work with your real back-end database. ACID compliance at production speed? Hm…). I also have customers that run on less than optimal networks and hardware. The idea that “the world works like we have GB internet and the latest m3 processor” is simply not true in general. Maybe someday, but it isn’t the world I currently live in.

so, subscriptions for read-only UI, or very special cases where you’re willing to pay the overhead is a nice to have. Using it as a default application wide is a very hard problem. I agree it is probably “the future”, and that future may be closer than I think, but it sure isn’t a slam dunk trivial task.

(I’m being very particular about deterministic functionality with ACID compliance…for a certain class of common application, the problems are easier and the bugs that come up are “tolerable”)

All that said, Nikita is a very smart person, and has written some great stuff. A front-end synchronized datascript database would be very cool indeed, and integration with Fulcro would be a very interesting thing… And in the case of firebase/instant kinds of solutions: many apps can use those with great success because you don’t have your own db server…you’re using their service (yet another business consideration…what if they go out of business???). If you can tolerate the risks of such a db-as-a-service provider, and you don’t have heavy transactional traffic on shared data, then my bet is they work splendidly

(and if they have CAS transaction support, you even have a good system for self-implemented ACID compliance)

I've been working on integrating Instant and Fulcro. there's some parts that fit very naturally, but also some that don't line up as nicely as it seems at first glance.

Fulcro's state is not a database, it's a normalized cache of state. different queries will overwrite parts of it, and that's okay. Instant query subs fire, the Fulcro state gets overwritten with the latest, and the UI updates.

Re the datomic query subscription: I assume you're using polling or tx-report-queue to get notified of changes? What techniques do you use to go from those corpse grained events to know which specific queries should be rerun? This has been something on my mind recently. I was looking at how the electric clojure folks are doing this and they seem to just run all queries on any change and check for differences. I'm doing something similar in prod but wonder what the next step is once scale steps up.

1. run the query, but patch the EQL to include :db/id at all levels 2. Remember that set of :db/id in association with that subs 3. Watch tx log. The content is EAV…the E’s are the :db/id that changed 4. Compare that set of E’s to the sets in subs: non-empty intersection == update. a. This works because new associations will add a ref to any parent of the new association 5. Refresh the set for (2) when you re-run the query. a. push the new result to the client

well answering my self transit-clj and transit-cljs are inconsistent

transit-cljs takes a string as second arg