Fork me on GitHub
#xtdb
<
2021-11-14
>
Tomas Brejla09:11:48

Hi. Do you have any suggestion on how to perform DB migrations? I'm wondering if something similarar to migratus/`ragtime` exists for (or can be adujested to work with) xtdb. What I'm specifically looking for right now is a way to automatically register my transaction function(s) at some point. For now, I'm just explicitly transacting the registration of the tx function manually when my app starts, but it's probably quite inefficient as the tx function hardly ever changes. So I'm looking for some better approach. Thanks.

refset13:11:26

Hello! I'm not aware of all the tools/libs that exist to help with this sort of thing, although perhaps @U01AVNG2XNFcan advise you on https://github.com/pariyatti/joplin.xtdb which uses ragtime, but if it's only transaction functions you're worried about for the moment then I would argue that there isn't really too much inefficiency in re-submitting identical versions of the same transaction function, since the document and EAVs are deduplicated, so the only storage cost is to the tx-log and the temporal index - both of which are relatively lightweight

Steven Deobald01:11:39

@U01LFP3LA6P fwiw, I probably wouldn't recommend using joplin.xtdb unless you're quite confident that's what you want. Pariyatti's usage of that library is paired with a pretty weird (for xt, anyway) schema-on-write thing which necessitates some clunky schema migrations. That said, I do think there's room for a proper ragtime plugin for xt, without all the baggage that joplin brings. I implemented that library before I found out that joplin is all but deprecated.

Tomas Brejla11:11:58

Thanks for the tips, Jeremy. I didn't know about the dedup functionality, good to know :thumbsup: . I guess that manual registering after node startup might be good enough for now for me. Still I agree with Steven: I think some sort of xtdb plugin for ragtime or similar might be a really good thing to have. I understand that there's definitely a way lower amount of db migration scripts needed for xtdb compared to relational (SQL) DBs as SQL DBs need DDL scripts for schema adjustments. But I can still see a need for occasional "data migration" scripts even in xtdb - and they should be handled in some sane, "standarized" way.

🙏 1
1
refset13:11:00

That's fair - I certainly don't disagree - thanks for the feedback! I'll give ragtime some thought and see what could be done 🙂

sheluchin18:11:31

If I have something like #{{:timestamp "2021-07-15T17:28:55.262-00:00" :count 15} ...}, how could I go about querying the data if I want daily, weekly, monthly, etc. counts? Is there something similar to the Postgres date_trunc function that will allow xtdb to perform the logic at the DB level and not in code? Is a custom aggregate the solution I'm looking for here?

refset23:11:16

There's nothing built-in to directly help with this. A custom aggregate probably could work well, but it wouldn't be inherently better than just doing it in code...although if you get it working in code then porting it to a custom function (+ subquery) or custom aggregate may not be much extra work