Query generally prefers AEVT

Using the A if it’s known provides a kind of clustering/locality akin to what a column-oriented db would give

but yes, worst case, you could still have so many ?pid , spread over such a long time (so that their entity-ids are not at all contiguous) that you fetch nearly N segments

partitions are a mechanism to control this

by enforcing a sort order

https://docs.datomic.com/on-prem/schema/schema.html#partitions

when you create an entity via a tempid, you can supply a partition; the partition id becomes the high bits of the entity id. By putting frequently-read-together entities into the same partition, you increase the chance that you will fetch significantly less than N segments for N items.

but this is not automatic, and you cannot alter an entity’s partition after creation.

Really interesting, thank you @U09R86PA4!

Curious question: Is there any database that solves this problem? Would love to learn how they approach it.

mature sql databases often allow you to partition rows according to some criteria. The point of this is to put like rows into the same physical storage silos. (It isn’t quite the same, but you can use it to solve the same kinds of problems)

e.g. postgres https://www.postgresql.org/docs/current/ddl-partitioning.html

There’s also often similar knobs on individual indexes.

Gotcha, thank you @U09R86PA4!