That’s the best I’ve seen.. and some transaction function helpers for upserting or conflict checking

Same problem here, no explanation.

uhg. it's affecting our production setup.. no bueno.

Thanks. Yes I have the same thing currently but it is pretty complex and can be difficult to debug so I’m exploring other options.

Also, it’s harder to convert these exceptions into friendly user-facing messages so that’s another reason to find an alternative. Right now I’m thinking a combination of a query in the peer and a unique attribute as a fallback

That is likely a bug; i will investigate

Ok, if needed I can provide you with a stacktrace

I see

sandbox.core=> (<!! (d/q {:query '[:find ?e
               :where [?e :db/doc]]
      :timeout 1
      :args [(d/db conn)]}))
#:cognitect.anomalies{:category :cognitect.anomalies/interrupted, :message "Datomic Client Timeout"}
sandbox.core=> 

when getting a timeout query

both with sync and async apis

what version of client?

(this is against cloud)

nevermind I’ve seen the behavior in another context; we will investigate

The exception is thrown after about 1 min, and the trace contains:

[
                "datomic.client.impl.local.Client",
                "connect",
                "local.clj",
                85
            ]

Solo or production stack?

production

I'm thinking this may be due to switching to on-demand provisioning on DynamoDb

I have no experience with that; I would hesitate to use it until we’ve tested it with Datomic

it would not surprise me, however

loading a database requires reading from dynamo to retrieve the log tail; if there’s a delay there (i don’t know how the new provisioning thing works) it would slow that process

in general the unavailable due to loading database will resolve and should be retried

Got it, thanks. What are the recommended Dynamodb settings fro production ?

erm. whatever was in the template 😄 I’d have to go look

Val ^ that’s a prod system i stood up yesterday

read @ 200 min, write @ 25 min

2000 and 500 max, respectively

it seems that slack isn’t letting me put in a screenshot

Great, thank you! I'm seeing the screenshot all right

Oh good. Slack sent me some message about file upload and free plan. shrug

If you need some write capacity upfront, because you're ingesting a ton of data, is it worth fiddling with minimum write capacity, or should you leave it well enough alone and wait for auto scaling to kick in?

The former

At least in my experience

Scaling will get you there eventually but if you know you're doing an import, you can bump it a bit and get a head start

Usually means less headache around making sure your import has long enough retry backoff etc

Ive rarely needed more than 300ish for a bulk import

Cloud uses a lot less ddb throughput than on prem

Good to know

Switching back to provisioned capacity solved it, thanks @marshall

Good deal.

Moving to prod will definitely increase your write throughput

you’ll be moving from a very small (t2.small) instance to a fairly substantial one (either i3.large or i3.xlarge)

you also get larger and faster caching on the local instance

which will make indexing jobs run faster

great, thanks

Yes, that is fine!