This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-10-03
Channels
- # announcements (4)
- # beginners (68)
- # boot (3)
- # business (20)
- # cider (39)
- # cljs-dev (7)
- # cljsjs (1)
- # cljsrn (12)
- # clojure (122)
- # clojure-brasil (2)
- # clojure-italy (7)
- # clojure-nl (5)
- # clojure-spec (60)
- # clojure-uk (41)
- # clojurescript (67)
- # cursive (7)
- # datomic (13)
- # emacs (6)
- # figwheel-main (18)
- # fulcro (40)
- # garden (3)
- # graphql (2)
- # hyperfiddle (4)
- # jobs-discuss (10)
- # lein-figwheel (5)
- # leiningen (12)
- # luminus (6)
- # mount (3)
- # off-topic (52)
- # portkey (2)
- # re-frame (1)
- # reagent (6)
- # reitit (24)
- # shadow-cljs (15)
- # sql (3)
- # tools-deps (12)
not really listening to this, but the video is gold !
Wondering if I'm getting banned for this
> you canโt repair that You can, if you mutate.
I wonder if TypedArray-backed persistent data structures could be operated on in parallel over the gpu with something like: https://github.com/gpujs/gpu.js
probably, but GPU support in JS is pretty primitive as of yet.
That's another hard problem with GPUs. AMD has pretty good OpenCL support, but CUDA blows OpenCL away as far as advanced features go.
And neither of those is supported super well from JS
I'm just starting to scratch the surface. It seems like a very cool idea to explore though.
Might be worth breaking out a C++ compiler and playing with it a bit there. The new NVidia GPUs (10xx series) have some pretty insane features, and C++ is the best supported language. This is my favorite example. CUDA supports unified memory, so you can allocate some data in the CPU and then just pass a pointer to the GPU. The GPU driver will page in main memmory as the GPU requires it:
the <<<...>>> thing says "this function runs on the GPU"
(from this article: https://devblogs.nvidia.com/cuda-8-features-revealed/)
Someday I'll get around to figuring out how to write a GC on a GPU, then I can write a lisp interpreter ๐
Heh, can't make them smarter when the key component is multiple threads reading the same data. Even refcounting doesn't work there.
what's the gist of the idea?
Basically, cooperative alloc/free, where the threads cooperatively clean up neighboring pointers when they clean up their own. And it gradually defragments essentially
But how does a thread know when a given pointer is no longer in use?
Well, I'm not yet sure it'll work, but I'm thinking the memory pool can have "top level" objects and sub-objects. And I'd only have to keep a list of links between threads and top-level objects. Threads would need to inc/dec top level object's ref counters. If a thread happens to dec one to 0, it is responsible for freeing the block.
And all the logic would be implemented behind the swap/deref interfaces for the irefs that point to the top level objects
I'd actually like to have a more thorough conversation with you about the details of the idea some time. Perhaps you could disabuse me of some of my misconceptions around memory management. I riffed a sorta neat, halfway finished wait-free array-buffer alloc/free mechanism though, which seems to enable some cool ideas.
sounds a bit like hazard pointers
Which I've thought could work fairly well with region allocation.
But yes, the devil is in the details
Aye, thanks for the hint there. That does seem like it's in the same problem space. I'll research that further.
one thing that can help here, is to implement the ideas using a linked list. A list of cons cells is about 10 loc in any language, yet hits all major problems involved with immutable data memory management
If you don't mind a fairly sloppy, thrown together video (at our Clojure meetup group), I touch on some of the general ideas here: https://youtu.be/G-VhNV5euSI The wait free alloc/free algo has been updated a bit since then though
Basically, it's a pairwise array of [[address][length],[address][length]... and calling alloc on the shared array buffer will just scan the array left to right. First to find wins. Freeing uses atomics to coordinate between threads.
Here's a version of the algo https://www.maria.cloud/gist/72fd24101ec3fac15c270a226576271d I have a slightly less broken version I haven't uploaded yet.
And I still haven't even started to test it in multi threaded situations, so it's still vaporware-ish. But the general idea seems to have enough shape now to look at and critique.
And I haven't fully articulated anywhere my ideas on how to do the ref-counting and the whole "top level" refs thing. I'll probably try to put together a design doc of some sort soon.
seen a few people mention wasm. google unofficially has schism, a scheme to webassembly self-hosted compiler: https://github.com/google/schism
was thinking of you specifically @U050PJ2EU
Yeah, I tried using schism, but the community seems so strongly divided on such major language issues... (that's a bad pun, BTW)
@andy.fingerhut what do you mean? hadn't heard about this
I have never used the project you refer to. I was attempting to make a bad pun on the name of the project (look up schism in the dictionary)
"We're sick and tired of your ism skism game" - Bob Marley