Hey @U15BH4U4V please can you share the config option map/file you are using? Is this using XT 1.21.0? Are you only running one node or are you attempting to run multiple nodes using the same/different config?

Thank your for replying @U899JBRPF. I kicked out everything and use the empty map at the moment to make sure I have no conflicting dependencies.

Hey again, to confirm, you are still seeing issues despite what you just wrote?

Yes, I had switched to an empty map before daring do post here ;)

Are there any potentially interesting log messages being printed? No "ingestion aborted" erorrs (etc.)?

Nothing I could find, though I switched xtdb namespaces to debug logging via the timbre slf4j appender. Definitely no errors.

The log of ok transactions vs. the http-kit triggered identical not ok transactions looked the same to me.

> It feels like threads working with a common tx-log but writing to different document-stores and indices. how is your start-node function being called? maybe there really are multiple unrelated instances being created accidentally. What does the attribute-stats API show?

Hello @U899JBRPF, I start xtdb

(defn mem-node []
  (let [node (xt/start-node {})]
... logging etc
    node))

This should always be the same and only node. I run xtdb version 1.21.0.

The described failure happens on a startup on a freshly booted machine.

I tested this to rule out that my debugging, which of course restarts with a fresh db, connects to old remnants (maybe off heap).

Thanks for the new details - I'm perplexed 😅

if a submit-tx succeeds, the transaction should be visible via open-tx-log if you sync and the latest-completed-tx should match the tx-id details of the submit-tx 'receipt' map

Have you checked those APIs also?

(last (with-open [l (xt/open-tx-log node nil true)]
                       (iterator-seq l)))

I'm surprised that example executes safely (and doesn't crash), certainly doing it for real you should be careful about consuming what you need from the seq within the with-open context, i.e.:

(with-open [l (xt/open-tx-log node nil true)]
  (doall (last (iterator-seq l))))

unfortunately I'm a bit lost trying to follow your descriptions here (undoubtedly my fault not yours though!), is there any way you could create a minimal end-to-end example of the behaviour, perhaps in a separate repository that you could share with me?

Yea. These tests just have two transactions on an empty db. One for setup and the second one is the one that fails 😉 It will take some time to think how to separate this one out.

cool, well I'm eager to help 🙂

out of interest, what are you evaluating XT against? Postgres?

The alternative at the moment is to use a datascript fork (datahike/datalevin) for state and create a transaction-log from the actually changed triples which these databases return when you transact. This can actually be done with about 50 lines of code but data wise one needs to think very carefully what one is doing since these db's use integers als entity ids. So if anything shifts one needs to be prepared. So xtdbs possibility to use {:person/id uuid} instead of an int looked enticing to me and would allow me to delete some code that translates between the two. My domain data has timestamps and is itself events so even though I like the temporal bits I had no need to use them so far. Also xtdb would theoretically allow me to have a triple as the entity which reduces my data modeling contraints.

Cool, thanks for sharing the context! DataScript is great for prototyping things...it was definitely my own gateway to XT 😄

The main architect behind XT actually started with a datascript fork also https://github.com/hraberg/datascript-mapdb

(although technically it's not a fork, since it's monkeypatching datascript as a library)

:-DD

Maybe we should call it a descendant or something ...

in xtdb, values are only shallowly indexed, so you can't efficiently query for map values. it's recommended to instead flatten the map, e.g. {:db/id 1 :co-ordinate-x 1 :co-ordinate-y 1 :co-ordinate-z 2}

The above answer is essentially correct, however if you don't the query to be fast (i.e. a full scan is acceptable) you can use regular Clojure functions like so:

[?e :co-ordinate ?c]
[(get ?c :y) ?y]
[(== ?y 1)]

ah, I was beaten to it ☺️ https://clojurians.slack.com/archives/C07V8N22C/p1661110562566609?thread_ts=1661105920.400009&cid=C07V8N22C

Thanks a ton