How hard would it be to integrate webworkers with go blocks, so that clojurescript had: (N go blocks : m (main thread or webworkers)) like clojure has if I understand (N go blocks : M heavy threads). I can see that one would need to have generalized serialization across the boundaries, but many blocks only communicate through output channels so if that was enforced for this pattern serialization and message passing would only need to work for data through the channel? Then one would need to do (how much?) work on core.async. Has anyone done this? What part is the hardest?

if you mean you just use (go ...) wherever and they run automatically in a worker then that is pretty much impossible

I mean use (go) wherever but enforce by convention that the go block only communicates with the runtime through output channels

if you mean you create a worker thats starts a bunch of (go ...) and they communicate over channels then thats trivial

I thought that made some of the problem easier but I'm not sure that's why I'm asking

I played a lot with workers and the gist of it is that the serialization cost outweighs all other benefits for most parts

unless you specifically design your workers to maintain most of the state and you only serialize small messages over channels

does that change with the sharedmemorybuffer thing?

nope

since clojurescript is fundamentally javascript objects underneath, sharedmemorybuffer could enable some zero-serialization solution? or do you in javascript already need to serialize things to get them into the buffer

yes, javascript also needs to serialize non-basic objects

can't just put any object into a SharedArrayBuffer, that only understands bytes

last stupid question: underneath javascript objects somewhere there are bytes, but the reason these bytes can't just be jammed over from thread to thread is that some of these bytes are pointers to things and there's no obvious way to know when to stop chasing pointers? like myJsObj = {prop1: "internal prop", prop2: external_ref} So the real problem is understanding the scope/circumferance of the object? If one had a clojure map which didn't point to anything volatile (data-only), if one had access to the bytes could one chase every pointer (because the scope is bounded), and expose those memory sections as read-only? So one could have zero-serialization read-only data sharing without having all memory shared? But to implement this you'd need to get access to the memory model and bytes of javascxript objects and a different type of SharedArrayBuffer (I think) and access to the memory model of the javascript engine (to expose them across a buffer-type-thing) which you don't without forking it Sorry getting a bit theoretical, last one

yes, you don't have access to the internal memory structures of the VM

(if that was an easy problem to solve they would have done that already)

but if one did, like I'm not about to do it but say you forked nodejs or firefox or lumo, read-only data-only copying could be done by selective memory sharing, which would be way more useful in clojurescript because it's already built mostly on operations on immutable data

*not copying, sharing

I'd say you have better chances of doing that by implementing a new VM 😉

I mean its all just bits underneath so in theory its all possible

thanks a lot for your insight