is this a legit query to see if an attribute was created as part of a tx? (vs an upsert) :

(d/q '[:find [?v ...]
         :in $ [[?e ?a ?v _ ?added]]
         :where
         [?e ?a ?v]
         [(= ?added true)]
         [?a :db/ident :loan/id]]
       db-after tx-data)

the attribute in this being :loan/id

I deleted a reasonably large database and ran "bin/datomic gc-deleted-dbs ...", but I terminated the process by accident after a short time. Pretty sure nothing actually got gc'ed, because storage size is about the same as before (postgres, and I manually did a full vacuum and restarted both transactor and postgres). Is this fixable?

@ckarlsen: if you can afford the down time, the guaranteed way to get the smallest size in storage is to back the dbs up, blow away the underlying storage construct (keyspace, table, bucket, w/e) and then create one and restore the dbs back there. Are you saying that additional calls to gc-deleted-dbs don’t accomplish anything?

@hueyp: not sure I follow the intended behavior of that query. Any ?e ?a ?v should necessarily filter to ?added as true if you’re not using a history db (only history has retractions). So even supplying and binding datom values in the query, the first :where clause should limit to assertions. Apart from that, it seems as though you’re checking for a hard coded attribute and whether it was in that transaction data, not sure what role using a query or db in that context plays. Is this in the context of with ?

@bkamphaus: luckily this was on my dev machine. Additional calls just yieled "no deleted dbs found"

Excuse me, may I ask a newbie question here: Let’s suppose I have 3M records in datomic database and I want to get first 20 sorted by particular field (checkin date for example, or whatever) since datomic query API doesn’t support ordering - does it means that I have to download all that 3M records to my peer and sorted them after?

@rmuslimov: if the sort you want matches the lexicographical sort order for the value when indexed (with :avet present), you can use index-range/`datoms` or seek-datoms to handle paging behavior lazily.

@bkamphaus: sorry, missed this … I’m doing a scheduled import and upsert’ing say 100 entities at once. I want to know which of those entities were created vs updated. I could look at the datums directly, but saw an example of querying tx-data here : http://docs.datomic.com/transactions.html … I think the only thing I use the db-after for is to to map the attribute to its ident (`:loan/id`) … but again, not sure

:loan/id is a unique identity

the thought was — filter tx-data to just added, look for attribute of :loan/id … get the value?

@hueyp: just off the top of my head, it may be a better option to get the create time of the entity (tx or t) and see if it’s the same as the tx or t? but that does imply doing the lookup for an entity in the log of a min aggregate, so I could see why it’d be desirable to just inspect the tx-data. But you’re really just looking at the tx-data then to see if the attr/value that woudl result in an upsert is asserted there or not? (it won’t be there in add form if the resulting tx was an upsert).

yah, I’m just looking at the tx-data to see if the attribute is present as a new fact

yah, trying out the example from : http://docs.datomic.com/transactions.html :

[:find ?e ?aname ?v ?added
 :in $ [[?e ?a ?v _ ?added]]
 :where
 [?e ?a ?v _ ?added]
 [?a :db/ident ?aname]]

which makes sense as you said ?added is always true for a normal db

so really I just want to filter against the tx-data and not bother with db-after I think

I was just liking the idea of using datomic query syntax vs map / filter

For what you're doing yeah - I would say that docs query is probably better suited for understanding less trivial changes.

thanks!