Fork me on GitHub
#babashka
<
2023-08-10
>
joakimen07:08:14

Been playing around with Executors/newVirtualThreadPerTaskExecutor in babashka, and it works great. Does anyone have any reference implementation on how to constrain the amount of concurrent threads when using virtual threads? Like a fixed thread pool, just with virtual ones. Reason I'm asking is because I'm working against an API that limits the amount of concurrent requests being done from one client

👍 2
borkdude07:08:07

I guess it's not a specific babashka question so you could also try #C03S1KBA2 or even StackOverflow (it's basically a general Java question) if you don't get good answers here

joakimen07:08:12

Agreed, just thought I'd start here since I've seen a few discussions about the virtual threads here earlier! Thanks

borkdude07:08:42

yeah no problem

jumar07:08:09

I would suggest you try to limit the number of such requests at a higher level, perhaps by using a bounded queue.

👀 2
jumar07:08:24

One simple usage of a bounded queue is in the source of tap>

(defn tap>
...
  (force tap-loop)
  (.offer tapq (if (nil? x) ::tap-nil x)))
where tapq is defined above
(defonce ^:private ^java.util.concurrent.ArrayBlockingQueue tapq (java.util.concurrent.ArrayBlockingQueue. 1024))

amithgeorge20:08:25

It is recommended to use Semaphores to control concurrency https://blogs.oracle.com/javamagazine/post/java-loom-virtual-threads-platform-threads Check the section "Three pieces of practical advice".

👀 2
borkdude20:08:34

nice! this class is already available in bb btw, so it should work, I think

littleli14:08:57

Usually you should not need to, you can run millions of them in fact. If target resource needs to be controlled for concurrency (like connection or so) semaphore is a good option.

littleli14:08:57

...also bounds can be set using structured concurrency pattern (StructuredTaskScope) I don't know if this is supported in babashka though.

borkdude14:08:56

not sure, but semaphore should work. countdownlatch could also work, but this is not in bb yet I think

littleli14:08:09

StructuredTaskScope I think it's a bit clumsy for use in Lisp. It's too javaish. But I can imagine some sort of macro that can do this in elegant Lispy way.

littleli14:08:25

This would be a nice research project thinking-face

mmer08:08:44

Hi, is there anything that can handle .tgz files in Babashka?

borkdude08:08:21

I'd just shell out to tar , it's available on pretty much any system including Windows