Fork me on GitHub
#xtdb
<
2023-03-27
>
rickheere12:03:58

Soon I'll be starting on a file archiving project. Basically the client revives a large amount of files every day and those files need to be categorized, metadata about the file collected and stored on disk. Because we want to use some s3 compatible blob storage layer it seams to be a good synergy with the Document Store part of XTDB in where we can use the same storage layer. The files that come in are from a couple of kb to sometimes ~1.5gb

rickheere12:03:42

I was wondering if it is possible to store those files directly "in" XTDB or should metadata and files be handled separately, metadatab in XTDB and files directly in the storage layer?

tatut04:03:05

that is the usual way imo, store files in S3 (for example) and metadata in XTDB

2
tatut04:03:31

you don’t really get any benefit from storing a large byte array in XTDB and only bloat the local indexes

2
rickheere08:03:33

Thank you, that was expected.