Fork me on GitHub
#datomic
<
2016-09-26
>
zane14:09:27

I'm also curious about the answer to this question.

robert-stuttaford14:09:17

i know that you can release a connection

robert-stuttaford14:09:00

the doc string seems to suggest that it's not designed for this though

marshall14:09:37

@zane @robert-stuttaford I’d be interested in what the use case is for that

potetm14:09:32

@marshall: re: https://stackoverflow.com/questions/39688899/if-you-discover-a-fact-after-the-fact-how-do-you-datomic/39690195#39690195 I think he's more interested in how to efficiently access historical values. I know datomic uses AVET for as-of lookups with datetimes. Is there a way to efficiently use that index to give you the datom at or immediately before a particular value? Or is there an optimized query for datetimes that does the same?

marshall14:09:12

You could use the log potentially

marshall14:09:25

I’m not sure what you mean by datom at or before a particular value

potetm14:09:21

Never mind. I was thinking for some reason that historical values would be in the live AEVT index.

potetm14:09:33

(Thinking was, you're given some date X by the user, you want to do an AEVT lookup to get the closest date prior to X)

potetm15:09:12

Ordinarily I would just use a query. Not certain what his use case is.

robert-stuttaford15:09:37

@marshall this person ( @bmays ) wanted to delete a time-sharded database and purge all its cache, and start a new time-sharded database. the question is, how to purge that cache

marshall15:09:08

I know what was asked. Why the need to purge the cache?

marshall15:09:27

If your new DB was created with a separate call to create-database

robert-stuttaford15:09:47

ah, my explanation was off: I’m noticing entities in the cache after restoring the database to a previous version.

robert-stuttaford15:09:00

his words are about a page up 🙂

marshall15:09:09

right, after restoring over an existing DB you need to restart peers and txor

marshall15:09:26

but if you’re putting in “separate” DBs, the cache shouldnt matter

robert-stuttaford15:09:38

yeah. so he's restoring, and that's why he's having issues

marshall15:09:04

you can use restore to put a ‘new’ DB (i.e. not a backup of the same db) into storage

marshall15:09:06

in a running system

marshall15:09:40

any time you restore a DB to ‘replace’ one in there, restart definitely required

robert-stuttaford15:09:12

i'm guessing because uuids

bmays15:09:08

@marshall thanks for the reply. The use case is that we have a ETL process for large data sets into different logical DBs that share a transactor. We don’t want to consume transactor resources to do the ETL so we want to simply ‘replace’ the database periodically. We’d like to avoid restarting the peers/stopping the transactor, because we have another database serving reads/writes to a web service.

bmays15:09:29

We can live with a JVM restart but it does feel like we’re abusing the restore functionality

marshall16:09:23

@bmays Are you replacing the database that serves the web service or a separate one? Do the same peers serve both?

bmays16:09:55

@marshall: The peers serve both and it’s a separate database.

marshall16:09:05

@bmays If you had dedicated peers to the DB in question I’d say you should take down the peer, delete the db, call gc-deleted-dbs, restore, then restart the peer. I’m thinking that won’t work well if the other (web service) DB is also served by those same peers.

bmays16:09:11

Gotcha, thanks guys

bmays16:09:12

We considered having dedicated peers for the DBs but it’s not worth it at the moment. We’re going to just do a staggered reboot of the peers post restore

bmays16:09:22

thanks for your help @robert-stuttaford

robert-stuttaford16:09:39

happy to learn along with ya!

zane18:09:10

That robert-stuttaford is all right.

misha22:09:01

greetings! is ~10 billion datoms datomic capacity limit includes history datoms (retracted ones)? or is it current db view only? (since those are being stored in a separate trees)

misha22:09:10

also, is it a datomic db limit, or datomic server limit (where multiple dbs will have to share those ~10b datoms)?