Hi. We are looking into the best practices for loading lots of data into Datomic Cloud. I have seen the documentation on pipelining transactions for higher throughput https://docs.datomic.com/cloud/best.html#pipeline-transactions. From that section, > Data imports will run significantly faster if you pipeline transactions using the async API, and maintain several transactions in-flight at the same time. The example that follows does not use the Datomic async API. Why is that? Should it use the async API to achieve higher throughput? Are there any additional best practices or things to look out for when loading thousands of entities into Datomic Cloud?

one key for batch imports is to always put retries in your transactions

@kenny

The example of transaction pipelining in the docs does not include backing off on retriable anomalies

I always do some back of the napkin estimation of # of transactions and number of datoms per transaction

good practice to label the transactions themselves with some metadata

[:db/add "datomic.tx" :db/doc "kenny did this, part 1/15"]

or have stronger idempotence markers in the DB metadata

Also still curious as to if I should be using the Datomic Cloud async api for maximal throughput.

Hello! I'm using pull in my Datomic query and want to blend it with an :as statement to change the name of something. My data looks like this: {:category {:db/ident abcd}} I can use pull do grab it with [{:category [:db/ident]}] which returns a structure like the above. I can rename :db/ident by pulling like this [{:category [[:db/ident :as :hello]]}] . This returns a {:category {:hello abcd}} however what I would love to be able to do would be have it return {:hello abcd} essentially renaming that whole path. I'm doing this within a larger query using pull. I tried pulling this specific part out of the query like this with the :keys option:

(d/q '[:find (pull ?e [:1 :2 :3]) ?ident
       :keys :nums :hello
       ...
       [?cat :db/ident ?ident]
       ...)

(d/q '[:find (pull ?e [:1 :2 :3
                       [{:category [:db/ident]} :as :hello]]) 
       ...)