Fork me on GitHub
#datomic
<
2021-05-03
>
Michael Stokley17:05:35

i apologize if this is a naive question. if a datomic transaction fails, for whatever reason, does datomic log the failure and the data it failed to transact?

ghadi18:05:00

what if the transaction failed to reach the transactor?

Michael Stokley18:05:39

i'm not sure whether this question is meant rhetorically? as in, would i still ask my original question if i had known to first ask, and then answer, yours?

Michael Stokley18:05:53

again, i apologize if my question is naive

Michael Stokley18:05:41

is there a version of my original question that is not naive? that you might ask about other databases?

ghadi18:05:58

I guess I’m trying to say that the domain of failures is pretty large

ghadi18:05:21

Are you trying to find some debug info on a previously failed tx?

Michael Stokley18:05:54

as part of a security threat model, we're trying to ask about user auditing. if we set up our system to audit user actions to create, update, or delete datomic entities (which we have, using the log-why functionality), and we fail to persist an audit record... yeah, would we be able to recover that data.

Michael Stokley18:05:27

my intuition is that this question might be about a write ahead log, but i'm not sure

Michael Stokley18:05:42

in our case, the audit log is atomic with the actual create, update, or delete

ghadi18:05:00

audit logs need to be in "user space"

ghadi18:05:09

datomic does not have entity/datom level access control

ghadi18:05:37

ostensibly the log-why stuff is transacted in the same tx as the CRUD

ghadi18:05:43

it will all succeed or all fail together

Michael Stokley18:05:16

perhaps since it will all succeed or fail together, i can push back on this question.

Dave A20:05:41

Trying to get my head wrapped around implementing a product recommendation system using Datomic. For example implementing a 'weighted graph' with a pre-calculated similarity score 'edge' between products. Or a "people who bought this item also bought these items" type approach. Anyone have experience with this or have any thoughts on an approach that might be workable?

Joe Lane20:05:36

Hi @UCURPS3GE, do you already have a Datomic system with this data or are you evaluating Datomic for a new system?

Dave A20:05:35

The latter -- haven't committed to Datomic yet, would like to use it for other reasons, but this one aspect I'm not clear on whether it would be a good choice

Joe Lane20:05:38

Datomic Analytics is a feature in both on-prem and cloud that allows for integrating Datomic with analytics tools (Python, R, Metabase, Tableau, Matlab, etc, ) and other services (Spark, Map-Reduce, etc.) by providing a PrestoSQL (recently renamed to Trino) connector plugin. This bridges the gap between Datomic as an ACID, transactional, system of record and the broader analytics / big data community by now supporting SQL (via Presto/Trino) which most tools support either directly or via JDBC. With Datomic Analytics, the tables are entirely virtual, meaning the data stays in a single place, and is only realized lazily upon presto queries. https://docs.datomic.com/cloud/analytics/analytics-concepts.html and https://docs.datomic.com/on-prem/analytics/analytics-concepts.html

Joe Lane20:05:34

As you begin to plan a system you should reach out to us via the support channels, we always enjoy meeting our customers and helping them succeed!

Dave A22:05:19

Thanks for the reply Joe. My product data sets are pretty well-defined, so I'm hoping I can get a reasonably good result with running graph-like traversals to find similar products in Datomic, vs. using a full-blown separate ML solution or service like AWS Personalize. This post https://hashrocket.com/blog/posts/using-datomic-as-a-graph-database provides some hope of achieving that with decent performance. Just wondering if anyone else has worked on this particular problem.

Joe Lane20:05:38

Datomic Analytics is a feature in both on-prem and cloud that allows for integrating Datomic with analytics tools (Python, R, Metabase, Tableau, Matlab, etc, ) and other services (Spark, Map-Reduce, etc.) by providing a PrestoSQL (recently renamed to Trino) connector plugin. This bridges the gap between Datomic as an ACID, transactional, system of record and the broader analytics / big data community by now supporting SQL (via Presto/Trino) which most tools support either directly or via JDBC. With Datomic Analytics, the tables are entirely virtual, meaning the data stays in a single place, and is only realized lazily upon presto queries. https://docs.datomic.com/cloud/analytics/analytics-concepts.html and https://docs.datomic.com/on-prem/analytics/analytics-concepts.html