Fork me on GitHub
#biff
<
2023-09-22
>
ianjones15:09:46

I asked someone at my work why they like datomic over xtdb and they responded “with datomic you dont have to build a database first before you start working on your app” lol Im interested to hear that comparison!

Jacob O'Bryant15:09:27

not sure what they mean by that, did they elaborate?

Martynas Maciulevičius16:09:12

Maybe they meant Datomic vs PostgreSQL?

Martynas Maciulevičius16:09:57

I'm interested to hear about Datomic's schema compared to just having your own chosen schema library with XTDB. Because the thing is that Datomic's schema is probably mandatory. So... "with datomic you dont have to build a database" doesn't really hold. But I didn't use Datomic and I don't know how it works. Maybe this argument could come from the fact that you have to develop your "own schema way". But then why not talk about schemas.

ianjones16:09:47

Im not sure I really understood their explanation either lol

😄 1
ianjones16:09:03

its was definitely datomic vs xtdb haha, i didnt mention potgresql

ianjones16:09:26

my only experience with xtdb is through biff’s wrappers and I’ve never used datomic so Im a n00b

ianjones16:09:10

one thing that im wondering about… datomic supports a many cardinality… and how to translate that to xtdb

ianjones16:09:44

is it just as simple as adding a vector field to your record of uuids from associated records?

Jacob O'Bryant16:09:52

yep, either a vector or a set

Martynas Maciulevičius16:09:53

I can do this:

{doc/data ["a" "b" "c"]}
And then look up by "c" alone. So you don't need 1-n table Edit: we both answered the same question. Edit2: Not only UUIDs but also strings and so on

ianjones16:09:08

I have to redo my data model lol I have a bunch of traditional (in the table centric sense) index records facepalm

🙈 1
Martynas Maciulevičius16:09:56

But sometimes it's good to have additional "tables".

Martynas Maciulevičius16:09:36

The actual difference between Datomic and XTDB for me comes from EAV datom-based indexing and Document-based indexing (in XTDB). This means that each transaction in XTDB will produce whole document or... nothing. So you can't act on a part of a document and reapply some updates "later in time" or replay what the insertion function did -- you have to settle everything during insertion. Biff provides some helper functions but in the end of the day you don't know why the attribute of a document remained the same in the database. I consider this a weakness of XTDB. And I still use XTDB. Edit: And they won't move away from this model in XTDBv2.

Jacob O'Bryant16:09:24

> don't know why the attribute of a document remained the same can you expand on that? if you want to find the point in time/transaction in which the attribute value was actually changed, you can use https://docs.xtdb.com/clients/1.24.1/clojure/#_entity_history

Martynas Maciulevičius16:09:12

If you want to change a title of something in the future then you COULD schedule that update in the future using :valid-from. This is allowed by XTDB. But there is a catch. When you change the title again now you have to perform "git merge" of two histories -- the previous future and the current future. Was the name change important? Why was it changed? Maybe I shouldn't change it? The thing is that these functions are not saved and maybe your title was derived from somewhere. But you can't know because now it's a discrete string. So you can't do it or you have to store metadata about it and... I couldn't really figure that out when I wanted to do it in my previous job 🤷 IMO I'd want to store tx functions. But this is not that XTDB provides. Also I don't even know how I'd do this in Datomic. I guess there wouldn't be a way to do it too but I'm not sure. But at least it would update one field in the future and not the whole doc. Because with XTDB it would actually "store" whole doc's snapshot. And maybe your title actually depends on the other "unchanged" attributes to not change. Then you also don't know what's going on. But as long as you don't schedule updates you're good. So I don't allow to schedule any updates and only work with "now" or past. And I don't rewrite the past. It's there for good.

Martynas Maciulevičius16:09:21

Basically if you want to publish an article in 5 hours you'll also reset it's title. So if you'll not rewrite your history with new title then you'll be in trouble. But then maybe you wanted to schedule the title change too? Haha 🎁

Martynas Maciulevičius16:09:56

So for me this is the problem with data model. There is also the problem that XTDBv1 has to fit into one node. But they'll address this in XTDBv2. So I wait for them to complete it.

Jacob O'Bryant16:09:22

oh, that makes sense... yeah, I've never actually used xt's bitemporality ha ha, so haven't run into this kind of thing. that does seem like a pretty good reason to at least not do document writes in the future, and just rely on some other mechanism for scheduling stuff (say, a :post/publish-at attribute + a cron job that looks for posts that are ready to be published?)

Martynas Maciulevičius16:09:58

I used bitemporality in my last job quite a bit. So... this is why I know 😄

🎅 1
Martynas Maciulevičius17:09:55

In Datomic's case you would be able to only publish the article's publish date without its title. So here is the place where EAV shines. I don't know how they handle history merging but I think that it won't be needed that much. But XTDB is simpler to reason about because it's just "put doc -- get doc". And tx functions are just returning docs. And then you're not forced into AWS. Datomic could be less optimized in tx execution get-doc sense as you'd need to query your doc. But regular people won't go into these issues any time soon.

Jacob O'Bryant17:09:48

datomic doesn't do bitemporality in any case, so you can't write datoms in the future anyway. but yeah, makes sense how EAV + bitemporality would avoid that problem

Martynas Maciulevičius17:09:30

Oh. I didn't know that Datomic can't write into future. Thanks! 👍

👌 1
Martynas Maciulevičius17:09:52

Then it may make sense to build a database of EAVs on top of XTDB.... hm.

Martynas Maciulevičius17:09:31

> makes sense how EAV + bitemporality would avoid that problem Partially. Because if you update the same attr with two different functions... you don't know what's happening. i.e. if you have only one code path that updates it then you're good with overwriting. But probably you don't have only one way to change your title. But it's easier to handle as you don't change unintended things.

👍 1
Martynas Maciulevičius17:09:25

But it's not a real issue for those that don't have this usecase. So it doesn't matter too much and it can be mitigated by different DB design (spoiler -- you still have to think how you design your database even with XTDB). For instance you didn't even think about inserting into future dates.

refset09:09:57

(late to the party, sorry!) I'd add that XT definitely doesn't currently directly solve any challenges of schema and therefore, yes, figuring out how to cascade 'updates' of individual attributes retroactively over time (whether from the past to the present, or from the present to the future) is left entirely up to the user, and it's non-trivial. But if you need to use these capabilities then the safest/easiest thing is to always process the entire history of an entity every time, and store sufficient schema/metadata info around each entity change such that there are no ambiguities about the (machine readable) intent

🙌 1
refset10:09:39

> “with datomic you dont have to build a database first before you start working on your app” > it was definitely datomic vs xtdb haha, i didnt mention potgresql I can broadly understand the sentiment, but someone can equally make exactly the same level criticisms of postgres vs datomic: "where's the query planner?", "how can I express complex logic and aggregations efficiently within a single query?", "how do I perform long-running transformations?" databases encode a huge number of tradeoffs and their merits should really always be evaluated in the context of particular use-cases

👍 1