Fork me on GitHub

Is xtdb somehow really sensitive to how far the SQL server is from the node? We are now setting up the cloud environments, and an ingress of data that takes 30 secs against local postgres took 20 minutes from the laptop to a data center some 100km away.


well I’d expect it to do lots of roundtrips and therefore be much slower than running the indexing in the same datacenter as the SQL server


We are doing the ingress as one big bunch of ops


not related to xtdb, but I’ve had a similar difference in SQL queries locally (in same datacenter) vs remotely via bastion… 90 seconds vs 11+ minutes


in oracle case it was greatly mitigated by tuning the fetchsize jdbc parameter


What is the typical number of docs per tx? The pipelining changes in 1.22 should improve the batching across small transactions quite dramatically


The ingress puts about 28k documents in one transaction

👍 1

One document in 28k transactions was quite a bit slower locally 😉


Maybe try to batch it and see if that makes a difference. (partition-all 100 txs)


Uf, just hit too. Partitioning indeed might make sense, since evict is scaling worse than linearly, so perhaps other parts are too

😬 1

(I tested evicting all those 28k documents and re-ingressing as opposed to just nuking the db, but I don't know if I have the stomach to wait for that eviction even on local rocksdb)


Hm, interesting. If I run several ingressions and evictions on the same node, the runtimes grow. First ingress was 11secs, partitioned evict 17secs, second ingress (same data) 155 secs, second evict 22secs, then closing and reopening the node, ingress 20secs, eviction 19secs


That was with the ingress in one bulk tx. Will try partitioning that next too


When the ingress is partitioned to 100 documents, it takes (with evict cycles) 11s, 20s, 19s and 15s. So it stops growing even with running node.


Ugh, the partitioning killed the perf on local postgres


All with 1.22, yeah?


With 1.21, still waiting for others to kick the 1.22 tires a bit

👌 1