Fork me on GitHub
#xtdb
<
2022-09-27
>
Hukka11:09:30

Is xtdb somehow really sensitive to how far the SQL server is from the node? We are now setting up the cloud environments, and an ingress of data that takes 30 secs against local postgres took 20 minutes from the laptop to a data center some 100km away.

tatut11:09:49

well I’d expect it to do lots of roundtrips and therefore be much slower than running the indexing in the same datacenter as the SQL server

Hukka11:09:07

We are doing the ingress as one big bunch of ops

tatut12:09:33

not related to xtdb, but I’ve had a similar difference in SQL queries locally (in same datacenter) vs remotely via bastion… 90 seconds vs 11+ minutes

tatut12:09:04

in oracle case it was greatly mitigated by tuning the fetchsize jdbc parameter

refset04:09:59

What is the typical number of docs per tx? The pipelining changes in 1.22 should improve the batching across small transactions quite dramatically

Hukka04:09:46

The ingress puts about 28k documents in one transaction

👍 1
Hukka04:09:27

One document in 28k transactions was quite a bit slower locally 😉

FiVo08:09:55

Maybe try to batch it and see if that makes a difference. (partition-all 100 txs)

Hukka08:09:32

Uf, just hit https://github.com/xtdb/xtdb/issues/1509 too. Partitioning indeed might make sense, since evict is scaling worse than linearly, so perhaps other parts are too

😬 1
Hukka08:09:21

(I tested evicting all those 28k documents and re-ingressing as opposed to just nuking the db, but I don't know if I have the stomach to wait for that eviction even on local rocksdb)

Hukka11:09:53

Hm, interesting. If I run several ingressions and evictions on the same node, the runtimes grow. First ingress was 11secs, partitioned evict 17secs, second ingress (same data) 155 secs, second evict 22secs, then closing and reopening the node, ingress 20secs, eviction 19secs

Hukka11:09:20

That was with the ingress in one bulk tx. Will try partitioning that next too

Hukka11:09:53

When the ingress is partitioned to 100 documents, it takes (with evict cycles) 11s, 20s, 19s and 15s. So it stops growing even with running node.

Hukka13:09:03

Ugh, the partitioning killed the perf on local postgres

refset16:09:05

All with 1.22, yeah?

Hukka05:09:04

With 1.21, still waiting for others to kick the 1.22 tires a bit

👌 1