2025-12-10 datomic | Clojure Slack Archive

datomic

itaied 2025-12-10T18:03:59.875519Z

i'm receiving this error while trying to import-cloud:

ImportingExecution error (ExceptionInfo) at datomic.client.api.async/ares (async.clj:58).
Response body did not conform to Datomic client protocol

execution:

clojure -X:import-cloud

; import-cloud.clj

(defn import-cloud
  [_]
  (let [file-config (-> "datomic-import-cloud-config.edn" io/resource slurp edn/read-string)]

    (println "Importing datomic data from:")
    (pprint/pprint (:source file-config))

    (println "into:")
    (pprint/pprint (:dest file-config))

    (println (dl/import-cloud file-config))

    (println "Completed.")))

deps:

:import-cloud {:extra-paths ["development/src" "development/resources"]
               :extra-deps  {com.datomic/local   {:mvn/version "1.0.276"}
                             com.datomic/client-cloud {:mvn/version "1.0.124"}
                             org.clojure/clojure {:mvn/version "1.12.3"}}
               :exec-fn     dev.datomic-import-cloud/import-cloud}

and config:

{:source {:server-type :ion
          :region "***"
          :system "***"
          :endpoint "***"
          :db-name "***"}
 :dest {:system "prod-replica"
        :server-type :datomic-local
        :db-name "prod-replica"}
 :filter {}}

datomic cloud version 9309 on aws what am i doing wrong?

Joe Lane 2025-12-11T16:13:36.180689Z

You can perform the import to a Datomic Local database from within the VPC, connecting directly to a query group node or primary-compute-group-node via the EndpointAddress in the Outputs tab of the CFT. Alternatively, you can query with the client-api directly to your cloud instances (either first trying to use the ClientApiGatewayEndpoint, and if that doesn't work, jump into the VPC and query against the EndpointAddress which circumvents API-Gateway all together) I keep suggesting you try to use the ClientApiGatewayEndpoint with the client-api first because jumping inside the VPC is a bit cumbersome and the import-cloud operation is actually attempting to import transactions in batches of 32 txes at a time. This means you may be able to identify the erroneous transaction by querying via the client-api for JUST that transaction because it may be under the size limitation of api-gateway.

itaied 2025-12-10T18:36:38.427909Z

i'm getting

"{\"message\":\"Request Entity Too Large\"}"

413 error how can i tell if the tx are too large?

Joe Lane 2025-12-10T19:45:06.851479Z

I believe this is an api-gateway exception

Joe Lane 2025-12-10T19:48:42.217239Z

Can you try again but omit :filter {}?

Joe Lane 2025-12-10T19:51:24.095999Z

Hmm. actually I don't think that will work.

Joe Lane 2025-12-10T19:52:41.094129Z

I'd open the local-db, find the latest tx that made it into the db, and then start importing 1 tx-at-a-time until you find the tx in the cloud system that was too large.

Joe Lane 2025-12-10T19:52:53.329319Z

Again, I think this is an api-gateway problem

Joe Lane 2025-12-10T19:53:12.687629Z

You could always create this db from within the VPC

itaied 2025-12-10T21:19:15.507509Z

yes i believe it's the 10mb limit of the api gateway what should be my go-to strategy to investigate prod corruption or bugs?

Joe Lane 2025-12-10T21:57:31.691259Z

Wdym prod corruption?

itaied 2025-12-11T06:47:41.960459Z

we have a bug in a specific state of data i'm not sure how the data reached its state in order to investigate i need to connect or download the production state and apply the failing transaction

2025-12-10T19:47:13.836169Z

Are there plans to move com.datomic/client-cloud off of v1 of the AWS Java SDK since https://aws.amazon.com/blogs/developer/the-aws-sdk-for-java-1-x-is-in-maintenance-mode-effective-july-31-2024/ after dec 31st?

Joe Lane 2025-12-10T19:47:42.850709Z

Yes

👍 3

🎉 2

benny 2025-12-10T19:54:24.698209Z

Is there a way to change the amount of jetty threads needed for the ping-host, ping-port transactor options? I found on my freshly provisioned small VPS that jetty wants 4 dedicates cores and my new VPS only has 4 cores total

✅ 1

benny 2025-12-10T19:55:37.280389Z

java.lang.IllegalStateException: Insufficient configured threads: required=3 < max=3 for QueuedThreadPool[qtp359672405]@15702a55{STARTED,3<=3<=3,i=3,r=-1,q=0}

Joe Lane 2025-12-10T19:56:21.411199Z

https://docs.datomic.com/operation/deployment.html#max-threads

🙌 1

✅ 1

Joe Lane 2025-12-10T19:56:57.388819Z

An annoying bug that we have a ticket for already

Joe Lane 2025-12-10T19:57:03.554419Z

Sorry about that

benny 2025-12-10T19:57:17.752829Z

Nah it's fine, I thought I looked there but somehow overlooked it

benny 2025-12-10T19:57:51.549479Z

Thanks for the quick help. Felt like a VIP at that speed. 🙂

Joe Lane 2025-12-10T19:58:42.511489Z

I do my best to ensure all my customers feel like VIPs 😎

Clojurians Log v2

datomic