Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2020-03-06

Channels

#datomic
<
2020-03-06
>
grounded_sage09:03:15

What would best practice be for say. Getting a CSV dump from a client each night. Which contains all historical data (that shouldn’t change) and active data that is changing. Committing just the changes to Datomic. Where the data they provide has ID’s which associate across other CSV’s. Obviously you could simply query for the set of ID’s in the database and then only transact the new id’s for the data that shouldn’t change. But some data is subject to change at varying frequencies. Whereas others shouldn’t change but the data may be messed up on their end and you also want to somehow capture that. Would you hash the data associated with the provided id per namespace/csv and transact that?

grounded_sage09:03:30

Framing my question is a bit difficult so I hope people can follow me

grounded_sage11:03:55

So I guess the model would be this set of data doesn’t change. If it changes transact it but only ever give me the first instance of it (then notify me it changed - probably auditing logic on my side). As for if they mess up their id space.. I guess that problem will surface downstream and we can retract the transaction?

grounded_sage11:03:04

Maybe something like. :ticket-purchase/active-data Boolean or :ticket-purchase/historic-data Boolean. Then when the data has changed when say historic is set to true. The data is logged with a false. So you can query the history of ticket id and see all changes even if it is wrong. But query the right one using the true.

vlaaad15:03:21

Is this a bug? 

(seq (d/tx-range (get-conn) {:start #inst "2020-03-06T15:14:52.000-00:00"}))
=> ({:t 38
     :tx-data [#datom[13194139533350 50 #inst "2020-03-05T14:24:23.642-00:00" 13194139533350 true] ...]}
    {:t 39
     :tx-data [#datom[13194139533351 50 #inst "2020-03-06T15:14:52.119-00:00" 13194139533351 true] ...]})
(entity 50 is :db/txInstant , so a request for txs since today includes tx from yesterday)

vlaaad09:03:18

Ah, I understood my mistake: start point 2020-03-06T15:14:52.000 is before the second returned tx 2020-03-06T15:14:52.119 (notice the millis), so it returns previous transaction which makes sense 👍

donyorm16:03:51

So I'm trying to setup of my project to work with ions in datomic-cloud, but I've been having issues with it crashing. I tried to reproduce locally, using the dependencies a ion push operation prints, but I'm getting the following error 

Error building classpath. Could not find artifact com.cognitect:s3-creds:jar:0.1.23 in central ()
Any idea why that library can't be found?

joshkh17:03:44

do you have the following in your deps.edn? 

:mvn/repos {"datomic-cloud" {:url ""}

joshkh17:03:44

do you get the error when you deploy your code? or does it happen when you run your project?

donyorm20:03:58

It happens when I run the project, but only when I include the list of dependencies given when you push. I don't have my terminal up now, but it's something like "overridden dependencies"

donyorm20:03:31

This is locally on my machine to clarify, but it makes it hard to reproduce the server environment.