Fork me on GitHub
#cljs-dev
<
2017-04-18
>
favila03:04:01

Im not sure how a webworker backed agent or dosync makes any sense given the js environment

favila03:04:52

The bigger problem which you don't mention is how the heck are we going to share code with a worker thread

favila03:04:31

The activity of workers needs compile-time precoordination

favila03:04:22

Also js has no blocking calls, so send-off makes no sense

favila03:04:48

Maybe send makes sense for CPU-bound stuff, but having to message pass with a worker is a big overhead

favila03:04:24

And I never need stm in Jvm clojure so I'm not hankering for it in cljs. Again message passing chatter will dominate any possible benefit from parallelism

favila03:04:16

Better abstractions are those that target map-reduce/fork-join like workloads, or something actor like for a stateful process

favila03:04:51

There is a cljs lib out there that tries to address the former style of problem. Web worker actor-like API is not something I am aware of

favila03:04:10

Web workers should be treated more like sub processes rather than threads. Any API meant to address threading problems is going to be a bad fit imo, that includes agents and dosync

john06:04:53

@favila Understood. Thanks for the input!

john06:04:56

The code sharing works like in servant https://github.com/MarcoPolo/servant-demo. you have to be careful with it, but it works. Post-compile-time closures will probably cause the most trouble. I'm hoping to have a rough proof of concept up in the next few days.

john06:04:59

and it may end up being the case that an agent abstraction isn't the right fit for (cl)js-land. I'm moreorless trying to spark some discussion and explore some solutions to multi-core computing with CLJS.

john06:04:04

I was referring to send-off as a primarily io bound operation, or long running service. Perhaps a logging service. Perhaps you want to run 10 or 20 goloops with some services, but you don't want to clutter up the execution of your UI thread...

john06:04:06

Whereas the send is mostly for lighting up all your cores at once, trying to get a job done as fast as possible.

john06:04:13

The STM piece is pretty speculative, agreed. And most people probably won't be running atomic bank account transactions in the browser 😉

john06:04:02

and yeah, if your job is less than 25 milliseconds, you'll be spending most of your time in traffic.

Roman Liutikov08:04:37

@john I didn’t read the whole thread, but in future these Clojure’s primitives and STM could be implemented in ClojureScript using SharedArrayBuffer (shared memory) https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer and Atomics (thread locks) https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics

Roman Liutikov08:04:58

SharedArrayBuffer is too low-level (raw binary data), but there’s possibility that once it is implemented and used in every browser, we will get more APIs for shared memory (at least that’s what I heard from someone who is close to SharedArrayBuffer proposal)

john13:04:42

@roman01la agreed. And I should be able to drop shared buffers in, once they're available. I'm already using Transferables to distribute ports out to the workers.

john15:04:41

We may be able to win back some time using persistent data structures by only pushing the updated nodes to the underlying persistent tree structure over the wire.

john15:04:49

For updates to a massive app-db in an SPA, that would be pretty important, if you want 10 workers updating your app-state.

john15:04:35

Cool thing about that method: it wouldn't matter if you were talking to a worker over a local channel, over a websocket to a worker on a server, or to a worker in a peer. Not for minified code sharing, obviously - at least, not without a bunch of work up front - but just for distributed data objects, CLJS's persistent data structures can do some magic for us there.