Fork me on GitHub
#datomic
<
2016-12-16
>
tjtolton15:12:02

So, what are the disadvantages to using datomic? What are the pain points and how are they managed? What kind of scaling problems does it have? When I attempt to sell datomic to my company, these are questions they will want to hear addressed

dexter15:12:26

the biggest limitation I found was that you can't push down processing to the nodes so with terrascale datasets you have to ship a lot of data. This can be mitigated by denormalizing so you just keep pre-filtered datasets around

pesterhazy15:12:55

some operations are definitely not fast by default

tjtolton15:12:57

terrascale datasets

tjtolton15:12:15

Good. that one is pretty far off, I believe.

tjtolton15:12:13

Also, can someone give me a link to a primer or a blog post on what the hell impedance mismatch is? This is one of those buzzwords that I keep hearing and only understand inductively

pesterhazy15:12:08

one downside is that you can't use your typical SQL-based analytics tools with datomic; if you're planning to use such tools, you need an etl job that exports data to an SQL database

tjtolton16:12:13

ahh, that's a really important one.

erichmond18:12:18

Is anyone from datomic sales / support here in the channel atm?

stuartsierra18:12:33

@tjtolton: when talking about databases, "impedance mismatch" usually refers to the fact that typical SQL + OOP systems have two different ways of representing information: relations (SQL tables) and graphs (objects).

stuartsierra18:12:17

When using a SQL database from an object-oriented language, there's almost always some translation that needs to happen between the database and your code.

stuartsierra19:12:15

As a consequence, you can end up with object-oriented code that breaks some assumptions about how objects should work (such as: calling a get method should be a fast, efficient operation) and also fails to take advantage of the features of the SQL database (joins, views, optimized queries).

stuartsierra19:12:21

Datomic tries to avoid the impedance mismatch by using a data model (tuples) that has a perfect one-to-one correspondence with regular programming-language data structures (maps and sets). The translation is automatic and loss-less.

stuartsierra19:12:32

On the other hand, this means that applications using Datomic typically stay close to the Datomic data model through much of the application code. It is unusual to wrap "objects" around Datomic transactions or queries, as is commonly done with ORM frameworks in object-oriented languages.

jaret19:12:17

@erichmond we're here. @marshall and I are here.

tjtolton19:12:43

@stuartsierra okay, that makes sense.

tjtolton20:12:43

so, the advantages of datomic in a read heavy system seem pretty obvious, because of local querying. Are there advantages to datomic in a write heavy system compared to, say, postgres? one that reacts to n million changes per day?

val_waeselynck20:12:23

@tjtolton your transactor limits the write throughput in a way that cannot be scaled horizontally (well, nor can postgres...). In addition, things can get challenging when your datasets get big (the famous "10 billion datoms" limit).

val_waeselynck20:12:50

@tjtolton on the other hand, reads don't slow down writes

val_waeselynck20:12:42

@tjtolton also, re: the impedance mismatch, I recommend this talk: https://www.infoq.com/presentations/Impedance-Mismatch

wei21:12:51

should you be able to use lookup refs for other entities within the same transaction? from my experiments it seems that’s not possible, and I’m wondering why not.

stuartsierra22:12:27

@wei Lookup refs in transactions are only supported for entities which existed in the database before the transaction.