This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-09-02
Channels
- # aleph (25)
- # announcements (17)
- # aws (2)
- # babashka (72)
- # beginners (44)
- # calva (6)
- # cider (3)
- # clj-kondo (109)
- # cljfx (1)
- # cljsrn (31)
- # clojure (151)
- # clojure-austin (1)
- # clojure-europe (36)
- # clojure-nl (5)
- # clojure-norway (2)
- # clojure-spec (17)
- # clojure-uk (12)
- # clojurescript (74)
- # cursive (57)
- # data-science (1)
- # datascript (28)
- # datomic (40)
- # depstar (15)
- # gratitude (3)
- # helix (3)
- # introduce-yourself (1)
- # joker (1)
- # kaocha (2)
- # leiningen (2)
- # lsp (70)
- # lumo (2)
- # malli (2)
- # meander (4)
- # off-topic (10)
- # polylith (27)
- # quil (4)
- # re-frame (18)
- # reagent (24)
- # ring (4)
- # rum (1)
- # shadow-cljs (102)
- # sql (2)
- # tools-deps (48)
- # web-security (8)
- # xtdb (5)
Is there an issue relating to memory leaks with aleph? When profiling with wrk2 I noticed terrible results which eventually got me to investigate gc and memory usage. Looks like ~50% of the heap is consumed by LinkedList, Deferred and ReentrantLock. Also looks like old gen keeps growing indefinitely. I can share more details and help with reproduction
Anyway, reproduction is easy run Aleph server with non blocking ring handler for 10 minutes at a high rate (I tried 10k qps), attach visual vm, behold the mess
The garbage can be collected once the benchmark finishes, could be perhaps some link between open connections and deferred-s not being collected?
I am not sure if this can be related: https://github.com/juxt/yada/issues/324
Zach just cut dirigiste 1.0.0, based off 0.1.6-alpha3. However, I don’t know who besides Zach can upload it to clojars. It’s not one of his repos that we added to clj-commons. Not sure if it fixes your issue, but fyi
You can ping me when he releases it an I'll give it another try. In the meanwhile, I tried passing an executor both to the server and the wrapper which creates the original deferred value. No luck.
@ben.sless My apologies, it’s available now. He released it to Maven Central, not clojars: https://search.maven.org/artifact/io.aleph/dirigiste/1.0.0/jar
ugh, still broken. I'm not going to hunt the cause down but if you need help reproducing it feel free to ping me
Hmmm, ok.
I'm sorry if it sounds like I'm abandoning it. It's just something I accidentally came across while benchmarking different servers. I don't use aleph and I don't think any of my colleagues do, either.
Oh, no worries!
I changed two things: • return byte array instead of input stream in muuntaja • did not inject match in reitit
@ben.sless If you have the time/interest to check it out again, I think I have a fix for the memory leak with ByteArrayInputStreams: https://github.com/clj-commons/aleph/issues/523#issuecomment-1075429913
I need to push the branch up then…
https://github.com/clj-commons/aleph/tree/bugfix/byte-array-input-stream-leak
@ben.sless i've seen a leak issue with manifold and stream/transform
- https://github.com/clj-commons/manifold/issues/197
yeah, it might not be the same issue - this was quite specific to stream/transform
. i played around in yourkit for a while - i could reliably duplicate the issue on our prod system, but never managed to get a minimal test-case, and in the end i ran out of time