Fork me on GitHub
#datomic
<
2019-03-27
>
quadron14:03:12

if in a domain every single event is timestamped (time being the integral part of every 'entity'), then is there a point in the database itself keeping a record of 'transaction history'?"

Joe Lane14:03:52

You have multiple time models, wall clock time, and logical time. Think of it as 2 axis on a graph.

Joe Lane14:03:21

If you don’t have a logical clock time you could never learn anything new about something in the past or correct any information.

Joe Lane14:03:30

That being said, datomic is a good fit for your transactional database, not as great a fit in the time-series analytics department when you have tons of datoms.

quadron15:03:21

does the fact that datoms are timeless in general, make the order of transactions dependent on the local ordering of a particular datomic instance? what if two datomic instances receive those datoms in different orders?

benoit15:03:32

@veix.q5 Datoms are not "timeless". Each datom points to the transaction that asserted it. The order of datoms in a transaction does not matter. Transactions are atomic.

benoit15:03:37

And there is only one "datomic instance", one transactor processing transactions serially.

johanatan06:03:23

How does this scale? What happens when a single transactor gets overloaded?

val_waeselynck07:03:22

The authors of Datomic made the explicit tradeoff of not making it write-scalable, and for many, many use cases that's fine. Systems tend to read much more than they write, and to give more importance to consistency than write throughput. Another DB making this tradeoff is VoltDB. Consider than, in typical RDBMS such as PostgreSQL, reads slow down writes (due to locking, required by mutability), making the typical OLTP workload even less scalable.

mdhaney21:03:59

If you eventually outgrow a single transactor, one thing you can do is split into multiple databases, each running their own transactor. On the read side, you can query across multiple databases (with on-prem only, Cloud doesn’t support this). The video below discusses how the Nubank guys designed their system with the anticipation of eventually splitting into multiple databases. https://youtu.be/7lm3K8zVOdY

johanatan21:03:14

@U0JPBB10W interesting, thanks!

quadron15:03:37

@me1740 what do you call a 'datom' before being asserted?

Joe Lane15:03:28

Volatile data

Joe Lane15:03:48

If it hasn’t been asserted it doesn’t exist as far as datomic is concerned.

👍 5
benoit15:03:05

@veix.q5 What joe said + a transaction is a collection of datoms https://docs.datomic.com/cloud/whatis/data-model.html#datoms

👍 5
quadron15:03:43

thanks for clearing this up, so a bunch of datoms are tied together in a transaction and datomic itself is the total ordering of those txs. right?

favila15:03:08

The easiest analogy is to git

favila15:03:23

A datomic transaction is like a git commit containing datoms

favila15:03:21

you can store a time value in the commit itself, but that is saying something different from the metadata about the commit itself

favila15:03:01

(this analogy breaks down a bit because datomic datoms all occupy the same data space, vs git where there's source and then another layer on top of it)

favila15:03:10

but generally everything you would use git for on your source you would use transaction metadata and the time features of databases (d/history, d/as-of d/since) on your data

favila15:03:39

This is an excellent guide to datomic's internals: https://tonsky.me/blog/unofficial-guide-to-datomic-internals/

favila15:03:22

And this is an excellent article explaining why you usually can't use transaction time as the only time in your data (or vice-versa) and why datomic is not a great fit for time-series data: https://vvvvalvalval.github.io/posts/2017-07-08-Datomic-this-is-not-the-history-youre-looking-for.html

❤️ 8
favila15:03:23

(the other reason is datomic is optimized for reads not writes)

souenzzo17:03:54

how to :find [?e ...] in datomic cloud?

dmarjenburgh20:03:20

The find-coll specification is not supported in the cloud version (yet?)

souenzzo18:03:12

there is docs about how to redirect cast to stdout (at my repl/localhost)

zalky19:03:38

Hi all, I have a datomic on-prem transactor deployed via cloudformation (as per the docs) and I'm looking into how to add classpath functions. Any advice or pointers to documentation?

shaun-mahood23:03:44

Any chance the correct solution for the problem at https://forum.datomic.com/t/dependency-conflict-with-ring-jetty/447/7 could be added to the Datomic Cloud docs? It didn't even cross my mind that there might be a conflict between these, and my normal workflow would be to check the dependencies or issues on github which doesn't really work here.

5
johanatan06:03:23

How does this scale? What happens when a single transactor gets overloaded?