thanks - we're in the process of migrating our docs site at the moment, although I can certainly put a redirect in

I am trying to construct a view of all changes that affected a customer-id in chronological order for audit purposes

This seems related: https://github.com/juxt/crux/issues/12

If you don't know when the changes are, then you will have to resort to scanning of some kind. It's possible we could expose something to help make it slightly more efficient but really the best answer is to update a dedicated customer entity each time some change that relates to it happens. Is that possible?

yes, my current workaround is to have a customer entity per customer, that has an :event. Then I use (crux/listen ,,, to filter on events I'm interested in and then :crux.tx/put a new version of that customer entity with the latest change embedded.

not yet sure if i can guarantee that events coming on crux/listen are only processed once or how to ensure I don't miss any

it might be better to not listen and instead create the event manually for every change, but that seems a little cumbersome

Hmm, so you are writing to Crux from different/independent processes here? i.e. one process that only submits events and another which does other things (including, currently, processing these events)?

yeah that's right

Listen events will be processed once per node so are unlikely to be appropriate for generating further puts, though I don't grok the use-case. It seems more likely that you want to be using a transaction function

i've just discovered that. in the process of refactoring now so that for every change i want audited, i also submit a copy of that change in the :event attribute to the customer-entity.

That should allow me to get a chronological list of changes with the history api. It's a little brittle, but I think it's fine for now. It's not huge amounts of data

Great, that sounds about right to me. Feel free to share a snippet if you would like reassurance about how brittle (or not) it is

A couple more thoughts to make sure my advice is sound: are you assigning each event a UUID id such that each event is its own entity? Are you always going to have monotonic events or will you need to be able to (re-)process events retroactively?

> A couple more thoughts to make sure my advice is sound: are you assigning each event a UUID id such that each event is its own entity? No I was going to just overwrite the same document

> Are you always going to have monotonic events or will you need to be able to (re-)process events retroactively? These will always be monotonic

Thanks for all your help Jeremy

thanks, good to know, and no problem! 🙂

@U899JBRPF I settled on something like this:

(ns api.endpoints.events
  (:require [clojure.spec.alpha :as s]
            [crux.api :as crux]))

(s/def ::organization-id uuid?)
(s/def ::entity (s/keys :req [:crux.db/id]
                        :req-un [::organization-id]))

(defn audited-put [node entity]
  {:pre [(s/valid? ::entity entity)]}
  (let [oid (:organization-id entity)]
    (crux/submit-tx
      node
      [[:crux.tx/put entity]
       [:crux.tx/put {:crux.db/id {:events oid}
                      :type       :event
                      :action     :put
                      :event      entity}]])))

one drawback is (I think) that I can only really update one entity per org per submit-tx, but I think that's fine for now. (Assuming you can't update the same document multiple times in the same transaction)

nice 🙂 although on that final line you probably want to have (:crux.db/id entity) though, otherwise the entity doc effectively gets stored twice in the document (taking up space)

you're right that you can't transact multiple versions of an entity at the same valid time (range) in the same transaction - the indexer only indexes the last version in the transaction. You can validate the behaviour like this:

(c/submit-tx node [[:crux.tx/put {:crux.db/id :foo-1 :a 1}]
                   [:crux.tx/put {:crux.db/id :foo-1 :a 2}]])

(c/entity-history (c/db node) :foo-1 :asc {:with-docs? true :with-corrections? true})

; =>
[{:crux.tx/tx-time #inst "2020-08-20T23:48:30.307-00:00",
  :crux.tx/tx-id 56,
  :crux.db/valid-time #inst "2020-08-20T23:48:30.307-00:00",
  :crux.db/content-hash #crux/id "453a7bd059ac67d80cf95b642c0773413ada1e52",
  :crux.db/doc {:crux.db/id :foo-1, :a 2}}]