Hello 👋 I have a schema to which I'd like to add a lot of :db/index true to improve query performance. I'm assuming this will trigger a job to add the entities to the AVET index. How can I monitor how long this job takes? I'm hesitant of transacting all the :db/index true in one go
You generally only add an index if you need it. Do you need it? Do you lookup or seek entities by value or have queries where the first clause is a range of values?
Thanks for answers! I like sync-schema because it could give me a basic idea of how long the database is being stressed.
About the real need for indexes, I thought of just adding it for as many attributes as I can, mostly out of lazyness. I'll do some basic research on the queries via with-redefs 🤔 .
why with-redefs?
Mostly to collect some queries in an atom and inspect them (hooking into d/q )
do you not have access to the code of these queries? or are the queries dynamic?
I do have access. I rather do that because I can browse around the most relevant parts of the website and see what queries are hit. Also yes, some queries are dynamic
You can call https://docs.datomic.com/schema/schema-change.html#adding-an-avet-index-to-an-attribute to wait until a new index is ready.
You might also want to watch the metrics on the transactor while indexing to make sure it's not overwhelmed.
> You generally only add an index if you need it. Do you need it? Do you lookup or seek entities by value or have queries where the first clause is a range of values? @favila I was just reading this https://building.nubank.com/optimizing-datomic-queries/ and noticed this query:
{:query '{:find [?album-name ?year]
:in [$ ?artist-name]
:where [[?artist :artist/name ?artist-name]
[?release :release/artists ?artist]
[?release :release/year ?year]
[(< ?year 1970)]
[?release :release/name ?album-name]]}
:args [db "John Lennon"]}
In it, it makes initial sense to use :db/index true for :artist/name . Would you also gain in performance by using it with :release/artists?not for this query; [?release :release/artists ?artist] where ?artists is bound will only ever use VAET; AVET provides no benefit and wouldn't even be consulted.
Oh, right! Thanks 🙏
even the :release/year clause doesn't use AVET
Well, that one may use EAVT, I guess? 🤔
datalog query prefers AEVT over EAVT
"a lot of db/index" may not make sense
you should paste the output of q with :query-stats true and :io-stats, if possible
it's probable that adding only one attr index matters
> "a lot of db/index" may not make sense
I mean I'll add to a lot of attributes of the schema :db/index true. Does that make sense? 🤔 I found that we are barely using :db/index and the https://docs.datomic.com/indexes/index-model.html#usage-notes state
> Attributes that are going to be queried by value need to be marked as :db/index true or :db/unique.