Fork me on GitHub

You really shouldn't cross-post unless you aren't getting an answer after a decent amount of time. Two minutes isn't very patient 🙂


Oops, my bad. Was kinda unsure if I should be posting here or integrant


It isn't a big deal. I suspect you won't get an answer until Monday "morning" US time in #integrant...


And I guess I was also thinking how I asked a slightly related question in #clojure earlier and this was my attempt at making it more specific (/another angle where I might get responses)


But yes, I will try to remember that! Actually, are there a list of rules/guidelines for clojurian slack? o:


There's a Code of Conduct, but it doesn't cover stuff like this. So we manually encourage folks not to x-post from time to time, and to be patient 🙂


Okay 🙂 thank you!


What is the correct way to parse / validate untrusted data in Clojure? In the past, I thought "user data => parse as edn => spec"; but now I am concerned about algorithmic denial of service attacks.

Linus Ericsson08:11:48

I would use malli because it allows closed specs. And be careful what I put in those specs of course.


Do you know how other languages/platforms handle such problem?


@U04V4KLKC: In languages with static types and serialization / deserialization generators, one can often get "size of struct + O(1)" validators; (though might still grow the stack too much, and not nearly as flexible as spec).


"Algorithmic" here as in: the attacker constructs input designed to maximize RAM / CPU usage.


given this:

(set! *warn-on-reflection* true)

(def http-client-1 ;; reflection warning: reference to field build can't be resolved.
  (.build (doto (HttpClient/newBuilder)
            (.version HttpClient$Version/HTTP_1_1))))

(def http-client-2 ;; no reflection warning
  (let [builder (doto (HttpClient/newBuilder)
                  (.version HttpClient$Version/HTTP_1_1))]
    (.build builder)))
http-client-1 generates a reflection warning, but http-client-2 does not. in my mind these are equivalent, but seems not. anybody know why the latter does not generate a warning but the former does?


I don't have an answer, and these are just observations rather than a suggestions, but I found it curious that if your http-client-1 innards are bound to a local first, it also doesn't reflect. It seems like something unusual in the behavior of def, but that is speculative.

(def http-client-1* ;; no reflection warning
  (let [client (.build (doto (HttpClient/newBuilder)
                              (.version HttpClient$Version/HTTP_1_1)))]

(let [client (.build (doto (HttpClient/newBuilder)
                       (.version HttpClient$Version/HTTP_1_1)))]
  (.sslParameters client)) ;; no reflection warning


@(promise) will suffice here I think (if you're just looking for a blocking call) - but on second reading, I think you want an operation within the loop)


Is there a way to explicitly destroy an object in Clojure? I have a repl session where I created some huge DS in memory. In the end only the GC was running. I niled the var with all the references to the data, thinking that would take care of it. It didn't. Does this mean I still have some lingering references to the data? Is there a way to resolve an issue like this without killing the repl session?

Alex Miller (Clojure team)17:11:29

the jvm does not have a way to explicitly destroy an object

Alex Miller (Clojure team)17:11:58

hard to say what exactly was happening without more info


This is what it looked like (visualvm heap size and clj-async-profiler during the end of the graph)

Ben Sless18:11:15

can you pop open the memory inspector, too?


I killed the session now.


can probably reproduce it

Ben Sless18:11:09

what kind of application is this? a web server? something else?


a crawler

Ben Sless18:11:31

hm. If you're doing a traversal there's certainly a chance you're holding on to stuff you shouldn't. Are you able to share the code? If not, recreating and examining the heap is also good


I actually know where the issue is. I was more interested in why I couldn't save the the session even after I stopped the crawler.


As Joshua mentioned outside of the thread it might have been *1 of which I did not think of at that moment

Ben Sless18:11:19

You could try to force gc with (System/gc)

Ben Sless18:11:45

that would at least tell you if you were still holding references

Alex Miller (Clojure team)17:11:21

could have been a lingering object reference somewhere - there are ways to debug that with either memory dumps or live profiling tools that do "path to root" analysis. or could have been something like a gc that had not yet found the dead object or was collecting but taking a long time

Joshua Suskalo18:11:19

on the repl if you def ed something you can use ns-unmap to remove it from your current ns. If you're on the repl then the last three return results and the last exception are kept around in the *1, *2, *3 , and *e vars. Once you've made sure your huge value is in none of these, you can call (.gc (Runtime/getRuntime)) and this will hint the garbage collector to run, which may collect your value.


True, I did not think about *1...

Joshua Suskalo18:11:40

If after all of that the object is still around, you probably have retained a reference to it some other way.

Joshua Suskalo18:11:01

But that said, there's still no guarantee that it will be removed.