Good question - as it stands, it'll use additional space in the block storage (we haven't yet taken full advantage of Arrow's columnar compression, it's on the roadmap 🙂). The way we've structured the indices, though, means that superseded data will eventually fall out of hot caches into colder storage, with the assumption that the object-store is significantly cheaper than storage on the nodes themselves. Definitely more to be written about it too 🙂

got it, will read-before-write then. still getting used to xtql:

(defn where-exists [m]
  (conj (reduce-kv (fn [acc k v] (conj acc (list '= (.-sym k) v)))
                   (list)
                   m)
        'where))


(rcf/tests
  (where-exists {:a 1 :b 2}) := '(where (= b 2) (= a 1)))

I expect the`backtick` library will be really handy with xtql, since there's so much un-namespaced syntax quoting required

(template (-> (from ::resources [*])
                    ~(xt-utils/where-not-exists {:weed/mount {:blah :blah}})))

heh, yep - we've inlined template as xt/template if that makes it easier 🙂

actually :assert-not-exists being a thing now, I guess we could lean on that given the transient tx log

oh that's nice, glad i'm not doing something idiosyncratic 🙂