Clojurians Log v2

yeah that's basically what he said in the conj keynote

you still want it to return on a chan

but he made it sound like a very straightforward change

go will still exist but not do the wild macro expansion; channel ops can then be blocking (on vthreads).

It sounded like Alex et al are trying to figure out a mostly API-compatible version of core.async that uses vthreads.

Now I definitely want to see that keynote:)

oh yeah they were going to add another macro as well that just explicitly puts you on a vthread

Not sure I'd categorize it as "a very straightforward change" tho' 😄

iirc he made it sound rather easy

might misremember

Was it in the Design in Practice in Practice talk? I thought it was in the Clojure 1.12 unsession?

yea the design in practice in practice talk, the keynote

the vod will reveal all

Oh, nice to hear they're working on this

Virtual threads are also still very new, jumping to then reduces the code in core.async a ton, as long as you continue to use them as if they were go blocks (no blocking) then it is great, but determining what actually is blocking is a lot trickier

I haven't looked in a bit, but at least earlier this year there were gists showing trivial deadlocking behaviors using virtual threads

really? I thought the only caveat was synchronized blocks

I would assume the deadlocking rules are the exact same as with platform threadas

It is possible it was synchronized related

that's more to do with pinning rather than deadlocking. but yeah, clojure lazy seqs used synchronized pre 1.12, but that's been patched

I think you can end up in situations where all the carrier threads are pinned (holding a monitor) but waiting on something from a virtual thread that cannot run

yeah yeah, that's the ultimate risk: that you lock up the state machine

(same risk in core.async)

the real risk for virtual threads in core.async is for code that clojure itself doesn't own using synchronized blocks

users are gonna have to have a pretty good idea of the libs they use

but apparently they're working on that at the JDK level

When I was at world singles I don't recall ever locking up the code.async threadpool, but we made a few attempts to switch to virtual threads for various things but I think newrelic related instrumentation code kept deadlocking

yeah the core.async threadpool size used to be 42 lol

but if you use them correctly, (offload i/o onto threads), you won't have any issues. much harder to avoid with virtual threads, that's fair.

Like, I get the impulse, virtual threads are cool, and doing it at the vm level solves all kinds of issues, but virtual threads are a huge change, and it is still very early for them

pretty sure if they address synchronized block pinning, there would be very few instances where they shouldn't be preferred. until then, I agree with you.

At Conj, I asked a room full of folks in the 1.12 unsession, who was using vthreads in production... One person put their hand up... and they said they've run into "a lot of problems" with them (as have we at work).

I would be on them if they would bump datomic cloud to 1.12 😄

I guess if you have to use vthreads, it would allow functions to be used which is the most annoying caveat right now, since you can't use map/reduce, etc. inside a go block But then your go code won't be compatible with non vthread implementations of core.async. And if you usurp thread, it similarly might not be compatible, as there are caveats to vthreads as well, and the amount you'd spawn, if you then ran it without vthreads it wouldn't be great. Etc.

why cant you use map/reduce inside a go block? or do you just mean the state machine would be able to park anywhere, regardless of call depth?

Ya. I mean like you can't take inside a map operation, because the go macro doesn't transform across function boundaries.

gotcha, yeah

I find it the most annoying thing haha. I'm ok being careful about where I do I/O, and also not parking deep inside a call stack, but since Clojure is functional, a lot of like conditional/loops are using functions where you'd want to take/put inside the HOF passed to them. And having to use doseq instead all the time is awkward.

And things like for expanding to HOF under the hood also don't work. It's what makes the GO macro feel like some restricted DSL.

Putting and taking with channels are as a much a side effect as anything else, so doing them in a lazy seq is not great

yeah I never wanted to put/take in map

mapv, transducers, doall, reduce, etc. It's pretty unconventional to be restricted to loop/recur and doseq.

The explanation is this: > In short, the go macro can’t do these operations without some serious work

I like it much better!

I think what it's saying is that. The way the go macro works, as I understand, is that it'll take what's left to do, and put it inside a function to execute later. And it doesn't know how to do that here. But if you instrumented the byte code itself, than map would just be a bunch of instructions in itself, and it could take those, wrap them, and put them to be executed later.

You'll have to track down tim and ask him.

Something to keep in mind (not related to whatever to the explanation is intended there) is channel operations are side effects, channels are mutable state, mixing side effects with lazy seqs like those produced by map can have very surprising behaviors

yeah, makes sense to me tbh, he even gave a reasonable answer: List<Promise>

regardless, it's not supported, which is also fine

Agreed about the side effects, just the explanation seems wrong (types, etc) I first read this a few years ago and had a "hmm" moment, and now I'm reading it again and wanted to see if anyone has a better understanding

Promises naively don't play well with the atomic guarantees in things like alts

Manifold takes the promise approach

idk, probably fairly straightforward to have a construct that does tho(?)

"makes no sense" just seems like an overstatement

could be wrong. wouldn't be the first time.

Oh, I am funemployed and working on a library

Concurrent ml has these things it calls events, which at first glance look like promises, but are not exactly that

The issue I have with manifolds approach is if you have two streams, do a take from each, so you have two promises, then do an alt on the two promises and a result from one, both streams still get a value consumed from them

recently learned about channels for java vthreads https://github.com/softwaremill/jox and I believe they also consume both channels inside alt/select, which is kinda counterintuitive (but I also may be just wrong, cause spent only 15 minutes on this)

I hadn't really looked at jox before, I appreciate that it's readme makes a distinction between a rendezvous channel and a buffered channel (even if they are sort of the same thing just configured differently)

core.async is missing https://github.com/softwaremill/jox/blob/main/channels/src/main/java/com/softwaremill/jox/Select.java#L325 (clean up after alts) which can cause problems with resource usages (memory leak like behavior with timeouts) so nice to see it