Fork me on GitHub
#xtdb
<
2021-03-21
>
nivekuil08:03:44

this line seems sketchy: https://github.com/juxt/crux/blob/master/crux-core/src/crux/document_store.clj#L38 it only works because (count ids) is always at least (count (keys cached-id->docs)), otherwise set/difference will try to use a seq like a set and explodes

👍 3
jarohen10:03:42

cheers @U797MAJ8M - fix pushed to master 🙏

nivekuil10:03:39

cool :) so if you know ids is always at least the size of the rhs, it's likely better to just (reduce disj..) the thing, the way set/difference is doing it now (or maybe do it better with transients?). that code seems like it may be a hot path on reads, so I'm curious if doing the additional set conversion would show up on your benchmarks. but I'm no expert, having spent like an hour failing to extend the thing to use an async interface -- that cache is tricky! segfaults galore

jarohen15:03:19

> I'm curious if doing the additional set conversion would show up on your benchmarks. came out well within our margin of error across our benchmark suite 🙂 will keep an eye on it between now and the next release

👍 3