Clojurians Log v2
core-async
Hi I’m definitely under-qualified to even be thinking about this, but I have to ask. What are the downsides of replacing core.async/go with something like this:
(defmacro go
[& body]
`(let [ch# (a/chan 1)
captured-bindings# (Var/getThreadBindingFrame)]
(CompletableFuture/runAsync
(^:once fn* []
(Var/resetThreadBindingFrame captured-bindings#)
(try
(let [ret# (do ~@body)]
(a/put! ch# ret#))
(finally (a/close! ch#))))
(Executors/newVirtualThreadPerTaskExecutor))
ch#))
Thread-pinning is one issue. Although Clojure 1.12 (Alpha 5) introduced changes to make pinning less likely in core code, there are still plenty of things "in the wild" that use Java's synchronized and could pin vthreads to O/S threads. The MySQL database driver is currently pretty bad for this. So a lot would depend on what exactly your go-using code is doing -- esp. if you are using Clojure 1.11 or earlier.
@seancorfield thanks for the explanation! Is there any practical reason to want to use these virtual threads?
There are plenty of practical reasons -- but the question is: what sort of work are you doing? vthreads work best for tasks with I/O (since that's where the park/resume hooks have been added in the JDK/JVM) rather than heavily CPU bound... but you wouldn't use regular go blocks for I/O-based work.
So there's a sort of conflict between how you are supposed to use go blocks and when you would want vthreads...
I use go-loops to wait for incoming messages for high-traffic websockets and process them once received. So it’s a mix of I/O and calculations.
The idea was to rewrite go and subsequently go-loop using vthreads, to use them. But I’m afraid of bugs 🙂
If not go-loops, what should I use for such work? @seancorfield
If you're using go-loops already, and it works, that seems... fine?
Thing is I’m not sure it works. For example I’m now debugging a problem, where if I’m creating 3 parallel workloads everything works, but when I add fourth everything breaks. Although they are produced by the same function, so the only difference I can imagine is the number of futures used under the hood.
If you're waiting on I/O in go blocks, that sounds like it runs contrary to recommended practices -- and you should be using threads instead.
So in your case, explicitly using vthreads would probably be a good thing -- but not by subverting how go works 🙂
Maybe something like promesa? It seems to have similar API to core.async, but uses vthreads under the hood
No idea. I've never used promesa.
Me neither. I’ll try and report 🙂
Ok, so promesa can be used as a drop-in replacement for clojure.core.async. It has a mostly similar api, so the changes required are minimal, and under the hood it uses virtual threads. It also has a better mult model for my case, mult can be created without a source channel and used as a broadcasting destination (a write-only channel, that normal channels can tap into).
As additional benefit is that it doesn’t color you functions and allows any kind of code within go-blocks, so it makes things a little easier too. @seancorfield
Good to know. I need to put that on my list to investigate.