Fork me on GitHub
#xtdb
<
2022-07-29
>
Godwin Ko00:07:07

when I’m using a map as :xt/id, I can’t get it back by using xt/entity, using string or number is perfectly fine, any clues for that? :thinking_face:

refset09:07:51

Could it be that something in your setup (e.g. editor plugins) is attaching metadata?

refset09:07:08

Metadata affects equality semantics in XT

Godwin Ko14:07:19

I’m posting through http api, so I suppose I can’t add any meta data into the map :thinking_face:

👍 1
refset16:07:56

Could there be some issue with keyword namespacing here also?

jmerrifield01:07:24

Is it possible to get an entity history with full TX operations for each entry? It would be great to have something like (entity-history node id :asc {:with-ops? true}) to be able to see what else was affected by each change to the entity in question. It seems like I'd have to open-tx-log with with-ops? and walk the entire TX log looking for the TX ids that touched the entity I'm interested in.

jmerrifield03:07:10

Huh I suppose I don't have to iterate through the whole transaction log thanks to after-tx-id so something like (.next (open-tx-log node (dec tx-id) true)) appears to get me a single transaction by its ID

refset09:07:47

Hey @U0EPG0ED7 , generally the recommendation here is to reify your transaction metadata into additional entities ahead of time, so that the necessary info is available via regular querying. Calling open-tx-log may be okay though, but might be slow/expensive depending on your specific tx-log choice

braai engineer11:07:28

Under what conditions would XT reads return but writes hang? There is only one (Java-based) consumer of the running XT node, backed by Postgres. Both XT and Java has been restarted. I.e. node.submitTx(...) hangs and node.awaitTx(…) throws TimeoutException even for 60 second wait time? My first thought was connection pooling limit because of unclosed node connections, but the caller has been restarted.

refset14:07:12

Postgres logging / monitoring may give some clues. It sounds like it is struggling to take out the necessary locks

braai engineer23:07:40

I truncated the tx_events table in the Postgres tx log and restarted the K8s pod to drop the Rocks index (no persistent volumes), and XT came back to life (I could safely drop all this data). There were 1,507,606 rows in tx_events. Could writes have been blocked because maybe node was in process of ingesting those events…maybe timing out somehow? Are locks still prime suspect? How do they work? I had previously restarted the node (w/o dropping tx_events) and that didn’t make any difference. I imagine if a node is busy ingesting tx log (no snapshotting), that reads would also be blocked?

refset00:07:15

Regular queries should generally work regardless of ingestion, but open-tx-log or entity calls could fail if ingestion-related connectivity is not working for whatever reason

refset00:07:45

I don't grasp how the locking works in huge detail but the jdbc module isn't massive to read and get a feel for, so I can recommend it

refset00:07:32

Locking varies a lot in practice based on the actual database backend being used, with different dialects

ossoso16:07:42

EDIT: cause was missing transaction function (see thread below). What are the steps necessary to investigate a transaction aborted error? I am getting from await-tx what seems like a confirmatory output {:tx-id 4, :tx-time #inst "2022-07-29T16:30:52.994-00:00"}, which is nevertheless accompanied by the log entry DEBUG xtdb.tx - Transaction aborted: {:xtdb.api/tx-time #inst "2022-07-29T16:30:52.994-00:00", :xtdb.api/tx-id 4}

malcolmsparks16:07:38

What kind of tx? A transaction function or a put/delete/match?

ossoso16:07:40

transaction function

malcolmsparks16:07:28

Wrap your tx fn code in a try/catch and in the catch block, return the exception details in a document with a fixed xt/id. I've got an example somewhere, one sec

ossoso16:07:04

Thanks, I get what you mean. I'm doing that already sans the try catch. Will add that now!

malcolmsparks16:07:56

Another common reason for a tx false is of your tx fn returns a document without a xt/id attribute

ossoso17:07:27

Looks like the transaction-fn wasn't being set up in my fixtures. With the addition of the transaction function everything's in order. I added the try-catch block to the function while I was at it so progress was made!

ossoso17:07:30

Thank you!

👍 1
1