clojure-norway 2024-05-27

@r.sevaldson viste meg en kul bruk av https://clojuredocs.org/clojure.core.async/pipeline-blocking her om dagen. Man kan putte inn hvor mye man vil at skal skje på én gang (“concurrency”), en input-kanal, en output-kanal og en transducer. Rett fram måte å utnytte flere kjerner, når man vil ha litt mer kontroll enn med https://clojuredocs.org/clojure.core/pmap.

👍 1

🙌 1

💡 1

Ivar Refsdal10:05:12

Eg brukte nyleg java.util.concurrent.Semaphore for fyrste gong:

(defonce ^:private max-threads-int 16)
(defonce ^:private max-threads-semaphore (Semaphore. max-threads-int true))
; 
(def ^:private wait-millis 3000)

(comment
  (if (true? (.tryAcquire max-threads-semaphore wait-millis TimeUnit/MILLISECONDS))
    (try
      (handler ...)
      (finally
        (.release max-threads-semaphore)))
    {:status 429                                          ; 429 Too Many Requests (RFC 6585)
     :body   {:message "Too Many Requests"}}))

For å unngå OOM på eit endepunkt. Før var det (i praksis) opptil 128 som samtidig kunne køyre/polle endepunktet

💯 1

teodorlu11:05:24

din løsning bruker backpressure, Ivar? Du sier bare til konsumenten “sorry, jeg kan ikke ta inn mer jobb nå, kom igjen senere”?

Ivar Refsdal18:05:51

Ja, om det tek meir enn 3000 ms å venta

👍 1

Jakub Holý (HolyJak)14:05:27

Hello, folks! I need to pick your 🧠 on how best to write a library that depends on JAR loaded at runtime. Namely, https://github.com/scicloj/wolframite depends on Wolfram’s jlink JAR, using its classes to encode expressions and process responses. Most namespaces have at least indirect dependency on these classes and fail to load w/o the jar on the classpath. Until now, we find and load the jar from your Wolfram installation automatically - but we want to make this heavy, side-effecting operation explicit, and replace it with an init! fn you call yourself. However, due to the previous point, this makes it impossible to load ± any namespace. Possible solutions: 1. Use reflection everywhere! Ugly, less error checking until you hit a code path, slower (but perhaps not meaningfully so) 2. Introduce a protocol abstracting away all jlink operations, extended to nil with a default impl. throwing exceptions, and having a global share state 🙀 holding the real implementation after you do call init! All code currently using jlink will instead use this interface, and access it through st. like @wolframite.runtime/instance . (I’d try to pass it as a param instead as much as possible, but not sure how far I get with that.) 3. A better solution??? WDYT? 🙏🙏🙏

cjohansen05:05:34

Is it not possible to get the required jar into ~/.m2 (e.g. via deps.edn) somehow?

cjohansen05:05:55

If it's proprietary code not distributed in a helpful way, maybe a workaround could be to create a tool that picks up the local jar, combines it with a barebones pom.xml and installs locally? clojure -T install-dat-local-shit

Jakub Holý (HolyJak)07:05:08

Thank you for giving this a thought! The contents on jar may differ based on the version of Wolfram you are using, and it indeed isn’t distributed in any useful way. Having to run an extra tool sounds more troublesome than side-effecting ns load we have now.

2024-05-27

Channels