This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-07-29
Channels
- # announcements (3)
- # babashka (47)
- # beginners (88)
- # calva (17)
- # clj-kondo (8)
- # cljdoc (1)
- # clojars (9)
- # clojure (98)
- # clojure-europe (53)
- # clojure-norway (2)
- # clojure-seattle (1)
- # clojure-uk (5)
- # clojurescript (20)
- # cursive (11)
- # data-oriented-programming (1)
- # data-science (3)
- # datahike (1)
- # datascript (3)
- # events (3)
- # graalvm (5)
- # honeysql (7)
- # hyperfiddle (1)
- # jobs-discuss (10)
- # leiningen (3)
- # malli (16)
- # music (4)
- # nbb (17)
- # off-topic (45)
- # pathom (9)
- # portal (7)
- # releases (1)
- # shadow-cljs (80)
- # sql (15)
- # tools-build (5)
- # xtdb (23)
when I’m using a map as :xt/id
, I can’t get it back by using xt/entity
, using string or number is perfectly fine, any clues for that? :thinking_face:
I’m posting through http api, so I suppose I can’t add any meta data into the map :thinking_face:
Is it possible to get an entity history with full TX operations for each entry? It would be great to have something like (entity-history node id :asc {:with-ops? true})
to be able to see what else was affected by each change to the entity in question. It seems like I'd have to open-tx-log
with with-ops?
and walk the entire TX log looking for the TX ids that touched the entity I'm interested in.
Huh I suppose I don't have to iterate through the whole transaction log thanks to after-tx-id
so something like (.next (open-tx-log node (dec tx-id) true))
appears to get me a single transaction by its ID
Hey @U0EPG0ED7 , generally the recommendation here is to reify your transaction metadata into additional entities ahead of time, so that the necessary info is available via regular querying. Calling open-tx-log may be okay though, but might be slow/expensive depending on your specific tx-log choice
Under what conditions would XT reads return but writes hang? There is only one (Java-based) consumer of the running XT node, backed by Postgres. Both XT and Java has been restarted.
I.e. node.submitTx(...)
hangs and node.awaitTx(…)
throws TimeoutException even for 60 second wait time?
My first thought was connection pooling limit because of unclosed node connections, but the caller has been restarted.
Postgres logging / monitoring may give some clues. It sounds like it is struggling to take out the necessary locks
I truncated the tx_events
table in the Postgres tx log and restarted the K8s pod to drop the Rocks index (no persistent volumes), and XT came back to life (I could safely drop all this data). There were 1,507,606 rows in tx_events
. Could writes have been blocked because maybe node was in process of ingesting those events…maybe timing out somehow? Are locks still prime suspect? How do they work?
I had previously restarted the node (w/o dropping tx_events
) and that didn’t make any difference. I imagine if a node is busy ingesting tx log (no snapshotting), that reads would also be blocked?
Regular queries should generally work regardless of ingestion, but open-tx-log or entity calls could fail if ingestion-related connectivity is not working for whatever reason
I don't grasp how the locking works in huge detail but the jdbc module isn't massive to read and get a feel for, so I can recommend it
Locking varies a lot in practice based on the actual database backend being used, with different dialects
EDIT: cause was missing transaction function (see thread below). What are the steps necessary to investigate a transaction aborted error? I am getting from await-tx
what seems like a confirmatory output {:tx-id 4, :tx-time #inst "2022-07-29T16:30:52.994-00:00"}
, which is nevertheless accompanied by the log entry DEBUG xtdb.tx - Transaction aborted: {:xtdb.api/tx-time #inst "2022-07-29T16:30:52.994-00:00", :xtdb.api/tx-id 4}
What kind of tx? A transaction function or a put/delete/match?
Wrap your tx fn code in a try/catch and in the catch block, return the exception details in a document with a fixed xt/id. I've got an example somewhere, one sec
Thanks, I get what you mean. I'm doing that already sans the try catch. Will add that now!
Another common reason for a tx false is of your tx fn returns a document without a xt/id attribute