when submitting a transaction with datomic pro, does a vector? not preserve its type on the transactor? I thought I remember a doc page about this but not finding it now. What checks should I be using for checking something is a lookup ref on the transactor? Does (and (seqable? current-value) (= 2 (count current-value)) (keyword? (first current-value))) look ok?
ah right, it’s in https://docs.datomic.com/transactions/transaction-functions.html#writing > Transaction data is serialized with Fressian. Transaction functions should not rely on, or presume, Clojure collection capabilities since collections deserialized by Fressian are guaranteed only Java interfaces.
am i correct that excision is meant to strip out all historical values as well?
i appear to have some historical values for an excised attribute on an entity, they are still available to (d/datoms (d/history ...) ...) , even after sync-excise on d/basis-t completes.
i didn't use before or beforeT : {:db/excise ID :db.excise/attrs [:attr1 :attr2]}
I am actually curious about this as well. Soon I'll be having to perform an excision for legal reasons and I assumed that excision rewrites history. Is the answer not that simple?
I've DM'ed you
Hi friends, we're slow-launching an upgraded hyperfiddle datomic browser: https://github.com/hyperfiddle/datomic-browser • Supports large Datomic databases and slow queries • upgraded entity browser UX with lots of new features • custom queries - any clojure query, i.e. supports Datomic ORMs and api wrappers, etc • prod ready, httpkit and jetty adapters, compatible with your prod middleware stack for auth etc • private beta, DM for a demo
wow!
So much awesome. I'm drooling.
scenario: excising PII from prod db restored to a non-prod environment.
would it improve indexing performance in any manner if the transactor didn't have any connected peers while it reindexed after a large excision (hundreds of thousands of entities with ~10 datoms each)?
what else could improve performance? would increasing writeConcurrency from 4 help? if so, what is a safe upperbound value to set it to?
update: found index-parallelism! is 8 the maximum? would a higher number make it any faster?
thanks 😄
What storage are you reading/writing to and what is your read-concurrency and write-concurrency?
AWS Aurora Postgres, and i believe we don't currently set a custom value for either of those, so it'd be the default!