This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-06-14
Channels
- # babashka (43)
- # beginners (47)
- # biff (1)
- # calva (16)
- # cider (7)
- # clerk (6)
- # clj-kondo (39)
- # cljdoc (49)
- # clojure (29)
- # clojure-brasil (1)
- # clojure-europe (93)
- # clojure-losangeles (1)
- # clojure-norway (34)
- # conjure (7)
- # datalevin (8)
- # events (1)
- # gratitude (3)
- # honeysql (6)
- # hyperfiddle (2)
- # introduce-yourself (1)
- # javascript (1)
- # jobs-discuss (9)
- # lsp (3)
- # malli (10)
- # off-topic (8)
- # pedestal (3)
- # rewrite-clj (2)
- # shadow-cljs (17)
- # sql (33)
- # vim (1)
- # xtdb (31)
i wrote my own version of pmap
using Executors/newFixedPool
to better suit my needs. it seems that criterium's quick-bench
doesn't play well with it, so how do y'all performance test code like this?
Out of curiosity, what happens when you try criterium on your code?
That said, criterium is basically running your code many, many times, so that (a) it is pretty much guaranteed that after the warmup phase that the JIT compiler has compiled your code to native code for your processor architecture, and (b) it runs it many times after that point to calculate not only the average run time, but also min, max, and percentiles, plus some other statistics I do not recall right now.
So the alternative is to do that via some other way, e.g. (a) run your code until you believe JIT is done, then (b) record min, max, and total run time for N runs after that, and calculate the average.
repl output:
[988.222s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[988.222s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-580-thread-6"
out of memory exception message:
; eval (root-form): (let [coll (range 100)] (user/quick-bench (pmap* inc c...
; (err) Execution error (OutOfMemoryError) at java.lang.Thread/start0 (Thread.java:-2).
; (err) unable to create native thread: possibly out of memory or process/resource limits reached
i'm calling (pmap* inc (range 100))
lol
I do not know the root cause of that, but it appears like it might be a case where running your library many times might hit the same error?
Try just running that call in a sequential loop, without using criterium, many times, and see if it also happens then.
oh that's a good call, thanks. i was using time
but that of course doesn't run multiple times
From the message, a guess at what might be happening: Something about your code's allocation of new threads keeps creating new ones, more quickly than it reuses old ones, and/or deallocating old ones, until the JVM runs out.
yeah, makes sense
> An unused ExecutorService
should be shut down to allow reclamation of its resources.
your comment made me go read the docs again, which i believe is the source of my issue lol
thanks for the rubber ducking, adding (.shutdownNow executor)
before returning solved the issue
"Zig is the highest-paid language to know this year (a new addition), while Clojure gets knocked from the top spot with a 10% decrease from 2022." Any guesses as to why? Are there just more junior/mid Clojure devs than there used to be? (A good thing! If that's the reason.) https://survey.stackoverflow.co/2023/#section-top-paying-technologies-top-paying-technologies
probably a combination of both more clojure devs and zig being really hot right now
yeah, I'd think the latter wouldn't explain the 10% decrease unless lots of Clojurians have moved to Zig? 😛
surprised at how fast Zig has blown up, I'm not particularly educated on the language, it initially just struck me as another C
seen Zig pitched around the place as "C without the bad bits" @U0482NW9KL1 seems there's plenty appetite for that
The fact that Zig compiles vanilla C and C++ is a huge boon for creaky legacy code. I can imagine this feature immediately increases its uptake.
Interesting that Zig 2023 average salary is still lower than that of Clojure from 2022 ($106,644)
i think it's more about that when the tech hiring ice age set in last year, "minor languages" like clojure took a hit, and then experienced clojure people just went on to found their own place
As an employer of Clojure devs (and dev myself) I can say that in my experience (n=1), many (external) Clojure developers we spoke to join our company have been asking for pretty high fees (sometimes without the shred of a proof of the relevant experience attached). It is/was a sellers market. It has gotten to a point where we made a decision to trial with educating junior developers to become Clojure developers, something which we did not intent to do at first. The downside of such an approach is that it takes time, the clear upside is that the developers we educate, get used to our culture and environment, and become valuable team members that like to work here. We pay our team well (I think/hope) and they seem to want to stay longer than those that ask for high fees coming in. We are very happy with that
I wonder if including Lisp as a separate language had something to do with it... was it like that previously?
I have yet to meet anyone who works in a non-clojure lisp who would be happy with inflating any clojure numbers.
FYI, this sort of discussion probably belongs in #C0KL616MN -- #C03S1KBA2 is intended for more technical or at least technical-adjacent discussions.
I do not know how representative StackOverflow's numbers are (nor were). Politically, the top slot is not a good place to be.
My bad @seancorfield! https://clojurians.slack.com/archives/C0KL616MN/p1686790624303589