Clojurians Log v2
datalevin 2026-10-03
I was hoping the 1.2.0 milestone would enable cardinality many attribute values with preserved order, but I don't see it happening from custom-data.md. Though there is a mention of "ordered list items" in the doc. Am I missing something?
Datalevin is so impressive. But I still find XTDB v1 the most ergonomic in terms of modelling regular clj data (but has storage, probably also perf trade offs). I see idoc enables maps as entity values, but is there hope of supporting clj vectors as values? To me that can be a real pain point with the datom approach.
To be more specific, in XTDB the following:
{:id 1 :attr1 [2 3] :attr2 [2 3 4]}
{:id 2}
...:attr1 would be indexed as a cardinality many ref type, but reading the entity (doc) itself preserves the transacted order: (:attr2 (xt/entity db 1)) => [2 3 4].
From the book:
> If you need an ordered or duplicate-preserving collection, model it differently: store a small typed vector as a :db.type/tuple when the positions and element types are fixed, store an arbitrary vector in an EDN-valued attribute, or model the list items as entities that carry a position attribute (a variant of the join-entity pattern in https://datalevin.org/docs/11-schema-design).
Tuples wouldn't work here, the other two suggestions could. But it would be a non-trivial amount of work (annoying compared to the clj version).XTDB v1 has a doc (entity) store and an index store. So it can do both. > XTDB automatically indexes the top-level fields across all documents as Entity-Attribute-Value triples. Vector and Set values are indexed as cardinality-many values, i.e. producing multiple EAVs. > You get cardinality-many relations from the index store, you get order preservation from the doc store. (This is why I mentioned storage trade offs. The flexibility is not free.) I would add it doesn't index values which are maps. (Those are indexed as EDN blobs and encoded via nippy.) So Datalevin wins there 🙂 Some work is needed because I want the attrs in my example to both encode graph relations and position. So in Datalevin I would need to introduce an attribute, in addition to the cardinality-many one, to store position (an EDN-valued one or one storing positions on list items themselves). I'm building a system where the user defines the schema and builds the queries, so these relatively minor data modelling annoyances have a way of becoming pain across the code base. I understand that this is a shortcoming of the EAV model, not Datalevin itself. I read this on the roadmap > • 1.2.0 extensible storage/query for arbitrary data. and thought it might fix this pain point for me. A Datalog database that just works with all fundamental clj data structures is the dream. With idoc, tuples and EDN values, variable length vectors/sets indexed as cardinality-many is basically the remaining unhandled edge case. Unfortunately for my purposes here 🙂 Thanks for taking the time to respond. And btw, the Datalevin book is a fantastic resource, thanks for that as well!
Is there a technical reason why something like the following couldn't work: • there's a schema opt that means "index this vec/set both as edn and individual elements" • transactions semantics: always tx whole vec/set, upsert • could have homogeneous (not sure about hetero?) value types inside the vec/set, e.g. ref type for AVE • Datalog queries would just work (?) • entity/pull/query fns would know to retrieve the edn instead of the set of individual items for these attributes ◦ hopefully with the item value types, we could pick out the edn EAV from the index? ◦ or edn might always be first/last in the index for that EA?
This is awesome! I see how idoc is the correct move for Datalevin. And I see it's already on master wow Thank you.