Fork me on GitHub
#clojure-dev
<
2019-08-17
>
andy.fingerhut04:08:06

OK, latest core.rrb-vector enhancement seems to get it back to O(log N) vector concatenate operations, but I haven't measured constant factors on it against Clojure's PersistentVector yet. Does anyone have any performance test suite for collections (particularly operations on vectors) lying around that I could start from?

dominicm04:08:54

I'm assuming you know the answer to this @andy.fingerhut, could you include a "why would I use this" section? I think I vaguely get the what of it, but I don't get what problem this data structure solves. Maybe I don't have that problem at all.

andy.fingerhut05:08:03

I can add that to the README, sure. Brief version: if you want everything Clojure persistent vectors are good for, but also O(log N) time concatenation of vectors (it is O(N) for Clojure's built-in persistent vectors, e.g. to do (into v1 v2)). One of the motivating use cases in the research paper was if you wanted to do parallel operations on contiguous chunks of a large array, each of which might have a result that is a different size than its input (e.g. filter, mapcat, etc.), and then quickly combine the results together.

Alex Miller (Clojure team)12:08:12

Zach Tellman has both a collection-check suite for property testing and some collection benchmarking code

andy.fingerhut12:08:07

Yeah, I was reminded of his large set of performance results that are published in his bifurcan repository. I will check those out.