Clojurians Log v2

We are so far primarily focused on the other direction, how to consume streams in a Clojure way (seq, into, reduce variants)

At least the reduce part is easy, I recall experimenting with extending the reduce protocol to streams. Maybe I got some edges wrong, though

The spliterator default support in Java collections allows you to pull streams from Clojure colls now, but those are naive (don't take advantage of the fact that Clojure colls are immutable and can be consumed in parallel), so we would like to also provide custom spliterators at least for vectors and maps. haven't worked on that yet.

https://clojure.atlassian.net/browse/CLJ-2775 is the seq/into/reduce candidate work

we've also identified some places where it might be useful to convert existing things to Suppliers and we might do some of that

@ben.sless reduce is actually subtle in a bunch of ways and we've now spent many hours talking about it :) • do you want access to parallelism if available? if so, you need not just a reducing function but a combining function. and the reducing function is not like a Clojure reducing function, it's an associative operator. we've decided we're not handling this (the apis exist and you should use them directly if so). we are narrowing scope to "Clojure style reduce over streams" and handling all the semantics to match Clojure reduce. • how do you implement the reduce? there are several mechanisms - iterator/spliterator, forEach, collect, etc. The only reason to do this is to expose internal reduce for stream impls that can do it faster than external iteration, and that really leads you only to the spliterator which is the heart of it. • what about reduced and early termination? the big thing you give up with internal reduction is external control (because the Java interfaces have no way to signal that ala reduced). What we are currently planning to do is to support Reduced (we have stuff that does that now and we want it to work on streams), and stop calling the reducing function. However, we have no way to early terminate, so the spliterator is still going to "visit" every value in the stream.

One benefit of the other work on functional interfaces adapters is it will be a lot easier to call a lot of the stream apis if you want to use them directly

I think I did it with iterators back when, around the transducers workshop I also tried extending reduce to CompletableFuture (images taken moments before disaster)

Thanks for the explanation!

yes, it works

we are using the same tech that javac uses to compile lambdas, which of course work just fine in Graal

while we are using invokedynamic, the method handle targets are static and known at compile time, so this is easy for graal to handle

I've tested it and can confirm that it works with GraalVM native-image. Not with SCI/bb at runtime (in scripts) though, this will be challenging to support since SCI doesn't use compilation

If AFn would implement all those functional interfaces as well, it would be a bit easier, but as Alex told me, this approach was messy (why?) and does not work for specific interfaces like FileFilter

the ones like *Function and *Operator require some type gymnastics with generics to properly implement at the same time, esp once you start looking at the default methods. And you can't handle the primitive variants at all there - those would have to go on the IFn$... We are also worried about this big increase in interfaces leading to ever more polymorphic type pollution in the runtime profiles, that's already an issue, this is likely to make it worse (possibly making all functions slower in aggregate). I don't have any evidence of that, but it is a concern. But not being able to handle other SAM types also seemed like a deal breaker.

there is inherent mismatch in that Java "functions" (instances of functional interfaces) are expected to have exactly one signature. IFn is inherently not that - it is multi arity and its inputs/outputs are unconstrained.