Is it a known bug that when there's a bunch of datums that get transacted simultaneously, it can randomly cause a :db.error/tempid-not-an-entity tempid '17503138' used only as value in transaction error?

It is often caused by one single entry that is the same structure as many others. Everything is fine, but for some reason, Datomic doesn't like it. Removing that one entry solves the problem.

Ordering of the entries in the transaction vector doesn't seem to matter either

The two datums causing problems:

[{:account/photo
   "REDACTED",
   :account/first-name "REDACTED",
   :account/bio
   "REDACTED",
   :account/email-verified? false,
   :account/location 2643743,
   :account/vendor-skills [17592186045491],
   :account/id #uuid "dd33747e-5c13-4779-8c23-9042460eb3f3",
   :account/vendor-industry-experiences [],
   :account/languages [17592186045618 17592186045620],
   :account/vendor-specialism 17592186045640,
   :account/links
   [{:db/id "REDACTED",
     :link/id #uuid "ea51184c-d027-44d0-8f20-df222e58daf3",
     :link/type :link-type/twitter,
     :link/url "REDACTED"}
    {:db/id
     "REDACTED",
     :link/id #uuid "c9577ca4-332d-41f0-b617-c00e89fc94b4",
     :link/type :link-type/linkedin,
     :link/url
     "REDACTED"}],
   :account/last-name "REDACTED",
   :account/email "REDACTED",
   :account/vendor-geo-expertises
   [17592186045655 17592186045740 17592186045648],
   :db/id "17503138",
   :account/vendor-type 17592186045484,
   :account/roles [:account.role/vendor-admin],
   :account/job-title "Investor"}]

[{:account/primary-account "17503138",
   :company/headline "REDACTED",
   :account/accounts ["17503138"],
   :tenant/tenants [[:tenant/name "REDACTED"]],
   :company/name "REDACTED",
   :company/types [:company.type/contact],
   :db/id "REDACTED",
   :company/id #uuid "ee26b11f-53ba-43f9-a59b-f7ad1a408d41",
   :company/domain "REDACTED"}]

During a meetup recording that I haven't uploaded yet I recorded my own maven private token from https://cognitect.com/dev-tools/view-creds.html is there a way I can regenerate that token?

Hey, I just missing something and can't figure out what. I am calling tx on datomic:

(defn add-source [conn {:keys [id name]
                        :or {id (d/squuid)}}]
  (let [tx {;; Source initial state
            :db/id                    (d/tempid :db.part/user)
            :source/id                id
            :source/storage-type      :source.storage-type/disk
            :source/job-status        :source.job-status/dispatched
            :source/created           (java.util.Date.)
            :source/name              name}]
    @(d/transact conn [tx])))

;; and then later API will call
(add-source conn entity-data)

Well, I guess I am going to do my migrations using a home-made solution now. I just lost all trust in Conformity. It doesn't write anything to the database most of the time I noticed.

Or are there alternatives?

can you describe your problem with conformity in more detail?

heya, coming here for a question about datomic cloud. I've noticed that while developing on a repl, I get exceptions as described in the datomic.api.client api:

All errors are reported via ex-info exceptions, with map contents
as specified by cognitect.anomalies.
See .

{:cognitect.anomalies/category :cognitect.anomalies/incorrect, :cognitect.anomalies/message \"Query is referencing unbound variables: #{?string}\", :variables #{?string}, :db/error :db.error/unbound-query-variables, :dbs [{:database-id \"48e8dd4d-84bb-4216-a9d7-4b4d17867050\", :t 97901, :next-t 97902, :history false}]}

@filipematossilva are you using the same API (sync or async) in both cases?

think so, yeah

have a ion handling http requests directly, and the repl is calling the handler that's registered on the ion

so it should be the same code running

we can see on the aws logs that the error is of a different shape

let me dig it up

on the aws logs, logging the exception, shows this

{
    "Msg": "Alpha API Failed",
    "Ex": {
        "Via": [
            {
                "Type": "com.google.common.util.concurrent.UncheckedExecutionException",
                "Message": "clojure.lang.ExceptionInfo: :db.error/not-a-binding-form Invalid binding form: :entity/graph {:cognitect.anomalies/category :cognitect.anomalies/incorrect, :cognitect.anomalies/message \"Invalid binding form: :entity/graph\", :db/error :db.error/not-a-binding-form}",
                "At": [
                    "com.google.common.cache.LocalCache$Segment",
                    "get",
                    "LocalCache.java",
                    2051
                ]
            },
            {
                "Type": "clojure.lang.ExceptionInfo",
                "Message": ":db.error/not-a-binding-form Invalid binding form: :entity/graph",
                "Data": {
                    "CognitectAnomaliesCategory": "CognitectAnomaliesIncorrect",
                    "CognitectAnomaliesMessage": "Invalid binding form: :entity/graph",
                    "DbError": "DbErrorNotABindingForm"
                },
                "At": [
                    "datomic.core.error$raise",
                    "invokeStatic",
                    "error.clj",
                    55
                ]
            }
        ],

(note: this was not the same unbound var query as above)

printing the error on the repl, we see this instead

#error {
                                      :cause "Invalid binding form: :entity/graph"
                                      :data {:cognitect.anomalies/category :cognitect.anomalies/incorrect, :cognitect.anomalies/message "Invalid binding form: :entity/graph", :db/error :db.error/not-a-binding-form, :dbs [{:database-id "48e8dd4d-84bb-4216-a9d7-4b4d17867050", :t 97058, :next-t 97059, :history false}]}
                                      :via
                                      [{:type clojure.lang.ExceptionInfo
                                        :message "Invalid binding form: :entity/graph"
                                        :data {:cognitect.anomalies/category :cognitect.anomalies/incorrect, :cognitect.anomalies/message "Invalid binding form: :entity/graph", :db/error :db.error/not-a-binding-form, :dbs [{:database-id "48e8dd4d-84bb-4216-a9d7-4b4d17867050", :t 97058, :next-t 97059, :history false}]}
                                        :at [datomic.client.api.async$ares invokeStatic "async.clj" 58]}]

that ^ is an anomaly

which is a data map

more precisely, (ex-data e) returns the anomaly inside that exception

ah, instead of ex-info ?

I imagine the datomic client wraps the exception doing something like (ex-info e anomaly cause)

we're not wrapping it on our end, just calling ex-data over it to get the anomaly

but on the live system, ex-data over the exception returns nil

which I think means it wasn't created with ex-info

I mean, I wouldn't be surprised if this is indeed intended to not leak information on the live system

that anomaly contains database ids, time info, and history info

just wanted to make sure if it was intended or not before working around it

@filipematossilva are you saying that you are not able to get a :cognitect.anomalies/incorrect from your failing query on the client side?

if by client side you mean "what calls the live datomic cloud system", then yes, that's it

@filipematossilva so what's different about your "live system" vs. the repl?

clearly it's an ex-info at the repl

I really don't know, that's what prompted this question

perhaps print (class e) and (supers e) in your live system when you get the error

or (Throwable->map e)

sync api or async api?

sync

regarding printing the error

I'm printing the exception proper like this:

(cast/alert {:msg "Alpha API Failed"
                   :ex  e})

do you have wrappers/helpers around your query? running it in a future?

on the live system the cast prints this

https://clojurians.slack.com/archives/C03RZMDSH/p1602169996347800

oh, yeah that's a com.google.common.util.concurrent.UncheckedExecutionException at the outermost layer

then the inner exception is an ex-info

on the repl, when cast is redirected to stderr, the datomic binary shows this

thanks. @marshall ^

just realized that the logged response there on the live system wasn't complete, let me fetch the full thing

ok this is the full casted thing on aws logs

understood

now that I look at the full cast on life, I can definitely see the cause and data fields there

which leaves me extra confused 😐

let me clarify:

in your REPL, you are getting an exception that is: * clojure.lang.ExceptionInfo + anomaly data in your live system you are getting: * com.google.common.util.concurrent.UncheckedExecutionException * clojure.lang.ExceptionInfo + anomaly data

where the Ion has the ex-info as the cause (chained to the UEE)

make sense? seems like a bug @marshall

to work around temporarily, you can do (-> e ex-cause ex-data) to unwrap the outer layer

and access the data

I can see that via indeed shows different things, as you say

but the toplevel still shows data and cause for both situations

I imagine that data would be returned from ex-data

let me edit those code blocks to remove the trace, I think it's adding a lot of noise and not helping

done

I think it's important to separate the exception object chain from the data that represents it (which may pull data from the root exception, not from the top exception)

Throwable->map for example pulls :cause, :data, :via from the root exception (deepest in the chain)

@alexmiller it's not clear to me what you mean by that in the current context

(besides the factual observation)

is it that you also think that the different behaviour between the repl+datomic binary and live system should be overcome by calling Throwable->map prior to extracting the data via ex-data?

root exception is the wrapped ex-info

you could do (-> e Throwable->map :data) to get at the :incorrect piece

I’m just saying that the data you’re seeing is consistent with what Ghadi is saying

Even though that may be confusing

ok I think I understand what you mean now

thank you for explaining

but the inconsistency is a bug 🙂

currently deploying your workaround, and testing

@filipematossilva this is in an Ion correct?

@marshall correct

in a handler-fn for http-direct

@ghadi I replaced my (ex-data e) with this fn

(defn error->error-data [e]
  ;; Workaround for a difference in the live datomic system where clojure exceptions
  ;; are wrapped in a com.google.common.util.concurrent.UncheckedExecutionException.
  ;; To get the ex-data on live, we must convert it to a map and access :data directly.
  (or (ex-data e)
      (-> e Throwable->map :data)))

I can confirm this gets me the anomaly for the live system

slightly different than on the repl still

live:

{:cognitect.anomalies/category :cognitect.anomalies/incorrect, :cognitect.anomalies/message "Invalid binding form: :entity/graph", :db/error :db.error/not-a-binding-form}

{:cognitect.anomalies/category :cognitect.anomalies/incorrect, :cognitect.anomalies/message \"Invalid binding form: :entity/graph\", :db/error :db.error/not-a-binding-form, :dbs [{:database-id \"48e8dd4d-84bb-4216-a9d7-4b4d17867050\", :t 97901, :next-t 97902, :history false}]}

which makes sense, because in the live exception the :dbs property just isn't there

but tbh that's the one that really shouldn't be exposed

so that's fine enough for me

thank you

is there are an official method to move data from dev-local to cloud?

Does anyone know how I get the t from tx (d/tx->t tx), but my tx is a map and the error in the conversion?

{:db-before [email protected], :db-after [email protected], :tx-data [#datom[13194139534369 50 #inst "2020-10-08T19:30:19.852-00:00" 13194139534369 true] #datom[277076930200610 169 #inst "2020-10-08T06:00:59.275-00:00" 13194139534369 true] #datom[277076930200610 163 17592186045452 13194139534369 true] #datom[277076930200610 165 277076930200584 13194139534369 true] #datom[277076930200610 170 17592186045454 13194139534369 true] #datom[277076930200610 162 277076930200581 13194139534369 true] #datom[277076930200610 167 #inst "2020-10-08T19:30:19.850-00:00" 13194139534369 true] #datom[277076930200610 168 17592186045432 13194139534369 true] #datom[277076930200610 166 #uuid "5f7f68cb-08f0-4cb2-964b-4e811a34a949" 13194139534369 true]], :tempids {-9223090561879066169 277076930200610}}

You need to grab the tx from a datom in :tx-data , in your case 13194139534369. I think something like (-> result :tx-data first :tx) will give you it

I think also (-> result :db-after :basisT) will give you your new t directly

Q: I want to store 3rd party oauth tokens in Datomic. Storing them as cleartext is not secure enough so I plan to use KMS to symmetrically encrypt them before storage. Has anyone done something like this before? If so, any advice? Or is there an alternative you would recommend?