Fork me on GitHub
#clojurescript
<
2022-07-30
>
john05:07:25

@zimablue we'd need to impl persistent datastructures over typed arrays, kinda like https://github.com/Bnaya/objectbuffer And to get that working well between workers, we'd need to do a simple form of GC, using weak refs to lean on the JS garbage collector

john05:07:11

A shorter path in the short term might be to wrap mfikes/bean around bnaya/objectbuffer, just to get a CLJS data facade over SABs

john05:07:03

Then you could legit share beans between workers with literal zero-copy

john05:07:27

But you'd still have one conversion step, serializing your native CLJS data into the buffer data. But after that, you could bang on it all you want, from different workers

john05:07:27

I'd recommend wrapping the buffered data in a swap!/deref interface, clearly demarcating the two types of data, so folks know that any data living inside of, or entering the swap transaction, is getting transformed to and from the formats, on the way in and way out. But whatever's living in there can stay that way, and can be shared between workers transparently

john05:07:59

There's an initial crack at Typed Array backed persistent vectors here: https://clojure.atlassian.net/browse/CLJS-1153

john05:07:11

As opposed to bnaya's global lock, we could use js/Atomics.compareAndExchange to only update parts of the persistent tree in a lockfree manner, like Clojure does with atoms with persistent data

john05:07:58

That's what I'm working on with tau v2. Just haven't finished the lockfree/wait free allocator piece. It's kinda mindbending lol

❤️ 1