Clojurians Log v2
clojure-dev
JDK 21 is here so we're going to see even more folks trying out virtual threads. There have been posts (here and elsewhere) about problems with synchronized and virtual thread pinning. The Clojure team are aware of this and are maybe looking at using ReentrantReadWriteLock in some places to alleviate this and make Clojure "more compatible" with virtual threads (see https://ask.clojure.org/index.php/12991/request-to-find-and-eliminate-virtual-thread-pinning?show=13004#c13004 for example).
At work, we're probably going to be moving to Jetty 12 "soon" and JDK 21 (we're already on JDK 20 with --enable-preview) and we'd like to see how Jetty 12's virtual threads option works... but it sounds like we could run afoul of the synchronized issues...
Is there a timeline for addressing this in Clojure? I'm assuming at this point that it's "too late" for this to be addressed in 1.12 but I'm curious whether it's considered a priority for 1.13 or perhaps a 1.12.x interim version?
We are going to discuss more this week (paused due to strange loop :)
Still possible we will do something in 1.12, could also do a 1.12.1. I did a bunch of assessment a few months ago
The stuff people have been posting is generally overkill but I think lazyseq and delay are probably the two highest priority things
Oh, cool. Thank you! I really thought we might have to wait for another full release cycle.
Adding onto what Alex said, it is a non-starter to add a RWLock to every lazyseq cell
Lazy seqs are mostly uncontended, and an extra object is a large allocation cost
We should endeavor to keep costs off the fast path
See lock-free techniques such as https://nullprogram.com/blog/2023/07/31/
Yeah, I trust you folks to make the right trade offs here... and I know this sort of thing is all about trade offs 🙂
Are people seeing thread pinning on lazy seqs that are not doing IO?
This ^^
Would really like to see actually problems rather than just “synchronized bad now” as that helps us make those tradeoffs correctly
Heh, well, once Temurin have JDK 21 builds available and New Relic supports JDK 21 and Jetty 12 ("some time next quarter"), we'll try out virtual threads in production and see what -- if anything -- breaks. But seeing folks post "I ran headlong into <this problem> with Clojure and virtual threads..." is why I figured I'd ask 🙂
Is that what they’re posting though?
I’ve only seen synthetic repros - please tag me if you see a real world issue
"I run a startup and we ran headlong into this synchronized block when using Java 19 virtual threads:" -- https://ask.clojure.org/index.php/12991/request-to-find-and-eliminate-virtual-thread-pinning
thanks, commented