> Can XTDB be backed by DynamoDB or Cassandra? not officially, but in principle 'yes' ...however trying to use them for the index-store could make queries unacceptably slow because out-of-process latencies stack up quickly YMMV > What happens with XTDB if I have larger database than my node's memory and its storage? RocksDB handles the first case (memory) well, and the second case (local storage) is largely why we decided to take a very different architectural route with the 2.x branch (though things like https://github.com/rockset/rocksdb-cloud are very interesting to consider in the context of pushing the 1.x design to its ultimate limits) > Also do you handle this differently between XTDB1 and XTDB2? yes very much - have you seen the talk Jon gave at the Conj this year?

> have you seen the talk Jon gave at the Conj this year? I've not watched it. I'll do it. Edit: Ok. I watched it and the Columnar bit seems to be the interesting part. But I remember that you posted in the forum that the in-node way of hosting the database will not be the primary way and also that value semantics will be SQL-like instead of JVM-like (e.g. dates). And this value semantics thing would mean that it's a different database. But it's good that the Datalog bit will be there and tables will be schemaless. I guess I'll go to see into the repository itself what's happening there.

Do you think that it would be possible to use XTDB2 without temporal functionality right now? How would one know which parts are already reliable-ish? As I understand once I'd import the data into the database the Apache Arrow bit should be stable as you didn't write that storage part on your own. Also if I'd want to reexport the data into XTDB1 I'd use a query the bitemporal query that would span throughout all time :thinking_face:

> Do you think that it would be possible to use XTDB2 without temporal functionality right now? It's quite pervasive, so you couldn't e.g. easily disable it to get better performance > How would one know which parts are already reliable-ish? talk to us 🙂 looking at tests is always a good idea > As I understand once I'd import the data into the database the Apache Arrow bit should be stable as you didn't write that storage part on your own. this is not quite the case currently, because while Arrow is solid we are still working on the exact layout of the Arrow files (essentially a primary covering indexing) to improve performance, so we can't support people trying to move between the changing formats currently - you would need to export and re-import each time

> Also if I'd want to reexport the data into XTDB1 I'd use a query the bitemporal query that would span throughout all time that sounds about right 👍

You could try Scilla :thinking_face:

You mean this? https://www.scylladb.com/

Her hair is really beautiful but she kind-of lost me when she said "you require less infrastructure -- which mean you get direct savings": https://youtube.com/watch?v=JPkrdWMVpPk

Yeah, my mistake, Scylla. It's supposed to be a high performance stand in for Cassandra

> high performance stand in for Cassandra I don't yet know how to use Cassandra/DynamoDB/Scylla effectively. In my mind I still want joins so I somehow have to mitigate this. I don't yet think productively in this consistency model yet :thinking_face: I know that I'd need to copy data and do my joins in advance. But I don't think I want to do it as this is overoptimization without real benefit :thinking_face:

I use scylla for my doc store and as the main database for data I don't care about joins/provenance in. xtdb1/2 are not suitable for write-heavy use cases but should scale very well for reads. and the consistency is fine, especially now that scylla uses raft

the dynamo-type databases are very simple, don't overthink it. it's just a key->key->value store