no, only after either enough rows or time, as you say. It'll persist any submitted transactions to the tx-log, so they'll be durable, but you won't necessarily get anything in the object-store just from a node closing

reason being that the blocks in the object-store need to be at deterministic intervals so that when you have multiple nodes, one node closing doesn't affect the others

Hm, then I suppose I had problems with the flushed txs getting read after (another) node starts again

ah - you may be running into https://github.com/xtdb/xtdb/issues/2789 ?

the marker of this one is that, if you submit a transaction (even an empty one) on the new node when it starts up, do you see subsequent queries returning data?

Indeed, that's what happens: empty transaction forces a sync

Or just a tiny sleep

cool, ok 🙂 openly, haven't committed to the behaviour of this one yet (opinions welcome!) • we could get every node to sync on startup (this'd increase startup time) • we could find a way for the node to store its own transient state locally (although this wouldn't apply in the 'immutable architecture' case of starting nodes completely fresh) • others?

this won't affect a client-server use-case - the clients all keep track of the latest transaction that they've submitted and ask every server they talk to to ensure that it has at least that one (so read-after-write consistency is preserved)

Have to see how the client asks it, that would quite sufficient. Used to calling https://v1-docs.xtdb.com/clients/1.24.3/clojure/#_sync in v1 too, just didn't see an API for that in v2

yeah - that's the one that is now each client's responsibility

the XT language clients do this for you automatically, so shouldn't be noticeable there

they intercept the return of submit-tx before passing it back, and then supply the latest one they've seen as the after-tx option to any queries

Ok, so I can get the latest from status, and use that then

what's the client btw, in your case? i.e. is it an ongoing process/browser session or similar?

cli

And also repl 😉

naturally ☺️

so CLI's starting a new process every time, I'd guess, bit trickier to keep that state

Yeah

Found a nifty little helper in xtdb.time called after-latest-submitted-tx

REPL - if your REPL connection is longer living than your XT node I might be tempted to keep some sort of submit-tx/q wrapper which preserves that state

Definitely, and keeping the node around is not hard either. It's just the first use that goes off the rails

otherwise, empty transaction is probably the easiest thing to do - at least until we've figured out how peer nodes should behave at first startup 🙂

I wonder if your CLI could store its latest-submitted-tx in a local file?

Sure, but sounds more complicated

yeah...

would you mind if the XT node sync'd on startup, btw?

even if it meant waiting for longer

I would often want it to sync, but then again I could foresee wanting to start submitting txs too and don't care about the sync

thanks - a good data point 🙂

ah - mind sharing a stack trace and/or the surrounding code, if you can?

Unfortunately it didn't output it, was just a print. I was loading a bunch of data in in batches of 10k. Looks like it got to 264070 with just 33 docs missing from what was in the dataset

I'll change things a bit so I could catch it

Running out of time though, so I have to get back to that next week

no worries - give us a shout 🙂