xtdb 2022-09-29 | Slack Archive

macrobartfast02:09:13

Does xtdb return only one document if there are numerous ones with the same values (but different ids) by default? I have numerous test documents with the same values (all x y z for latin-name, common-name, img) with different random ids, and querying them only returns one…

(defn query []
  (xt/q (xt/db dbnode)
        '{:find [latin-name common-name image]
          :where [[e :treefrog/img "z"]
                  [e :treefrog/img image]
                  [e :treefrog/latin-name latin-name]
                  [e :treefrog/common-name common-name]]}))

(PS - open to corrections on how I’m constructing my query, too! Here I’m trying to find all documents with the value for :treefrog/img of “z”)

refset03:09:32

Yep these "set semantics" are intentional and really what Datalog is all about. If you want to see all the duplicates then you have to :find (and return) the ids also, otherwise the result tuples get deduplicated

macrobartfast03:09:37

aha ok, all’s well then. I suppose people prevent duplicate document creation, then, in software.

macrobartfast03:09:13

In the case of ‘same data, different time’ a timestamp would make sure one gets all the documents, I’m guessing.

✅ 1

refset03:09:31

Yep exactly, and there's an example of pulling out the timestamps here: https://docs.xtdb.com/language-reference/datalog-queries/#custom-functions

Mikko Harju17:09:58

Hi! I'd be interested in hearing any migration strategies that people have found handy with XTDB. I would find it pretty trivial to just append migration logic to db-component startup, one function at a time with version checks (kind of an ad-hoc Liquibase specific to this project and baked into code...)

🙌 1

Steven Deobald23:09:51

I wrote https://github.com/pariyatti/joplin.xtdb ... and shortly after realized it was a mistake. 😉 Largely, it doesn't even really handle "migration" so much as storing a schema which dictates writes into the db itself. It also restricts XTDB values to a set as closely resembling Datomic's as possible.

Steven Deobald23:09:09

Since I'd describe migrations in XTDB as "mostly a mistake", I guess I'd jump back a step and ask: what's your desired outcome with migrations? Are you just hoping to keep a consistent schema across an entire timeline? And if you do, are you okay muddying the waters of what qualifies as "historical"?

Mikko Harju04:09:29

Yes – good thoughts! I was going to add that the migrations I'm mostly talking are data migrations – I need to modify the existing database with new information and possibly make transformations to existing documents taking this new data into consideration.

Steven Deobald10:09:25

Ah, interesting. Your current approach sounds sane, and not unlike the kinds of schema/data migrations I've seen in mobile apps for SQLite dbs. That said, if you wanted to extract a tool (however small) for data migrations over xtdb, I sincerely doubt you're the only one who's wondered this. 🙂

👍 1

tatut04:10:52

It feels like the bitemporal aspects bring another point to consider… do you want to migrate docs that “from now on, the docs look like this” or do you want migrate all histories “the docs have always looked like this”

tatut04:10:42

changing the shape of your data means your code needs to know when the change happened, when you are querying in the past

Mikko Harju04:10:10

Excellent point! What changes here are references to other documents – basically querying with proper valid times should make queries it consistent across time.

2022-09-29

Channels