Clojurians Log v2

Is that incremental materialized views that you are talking about? It is in scope prior 1.0 on my roadmap

Query subscriptions are similar to incremental views, yes, but incremental views are also efficiently maintained, which I would say is more sophisticated than knowing when to recalculate. Index listening is a more basic building-block feature, though.

Or file an issue describing what the API looks like. Thanks.

Will write something down, but I would allow the user to listen on indices with the same semantics as d/datoms. Of course, the holy grail would be a continuous query (subscription?) that re-runs only when the indices that affect a query change (similar to Dataflow). Like qseq but emits batches of entities as new data arrives. Would need lazy datalog engine that blocks. Difficult to implement. E.g. in Electric Clojure we commonly watch a Datalevin conn and rerun queries, but that means if User 123 transacts something that only affects their data, queries rerun for all 1k logged-in users that share a database (you can shard but that has costs), even when their data has not changed. (see https://github.com/sixthnormal/clj-3df) So an index subscription is a way to approximate query subscriptions because I can put an attribute on entities pertaining to a given user and listen for changes on that :ave index, and then re-render that user's views with fresh data. This can be done at an eventing layer, but then I need to re-implement a Datalog engine. E.g.

(def listener
  (d/qsub
    '[:find ?invoice
      :in $ ?user
      :where
      [?invoice :invoice/user ?user] ;; this query only has to trigger when [:ave :invoice/user user-id] fires.
      [?invoice :invoide/date ?date]
      [(> ?date #inst "2024")]]
  db user-id
  (fn [tx-data] (prn 'changes-here-can-trigger-re-render)))

https://github.com/juji-io/datalevin/blob/c7c249e99d9f90f0dd592afb3641fb6f2b60d0d5/src/datalevin/core.clj#L941

listen! is what you are looking for

Thanks! Would be nice if there was a way to subscribe to particular eids or attrs at the engine level (post-transaction), so I don't have to discard txes for the entities I'm not interested in.

Yeah. It would be. Current API is still pretty flexible in my experience. For simple filter on events or attributes I do something like this:

(d/listen! db :message-listener
  (fn [{:keys [tx-data]}]
    ;; Only fire if a transaction sets message content
    (when (some (fn [[_ a _]] (= a :message/content)) tx-data)
      (broadcast-refresh-to-connected-clients!))))

Sure, PR welcome