^long works the same. But Thread/sleep now has an extra overload with 1-arity, and that switches on parameter type matching.

right, but then I would expect reflection warning, not Syntax error caused by "Unable to resolve classname: clojure.core$long@..." exception

From syntax perspective those expressions suppose to be identical, isn't it?

Couple other things worth noting: • Vars can't store primitives • Tags on vars are resolved - they aren't preserved as symbols (that's why you see clojure.core$long@26a94fa5 instead of long) That means, (def ^long foo 1) is actually incorrect. Regardless of the Java version.

aha, so that's why there is "Unable to resolve ..." exception: ^long tag got resolved into the reference to clojure.core/long but when reflection tries to find a proper method of Thread/sleep this reference does not point to any class. So this could happen in any JDK version and I hit it only because after JDK 19 Thread/sleep needs to use reflection. Do you know any place where I can read more about the topic? https://clojure.org/reference/java_interop#typehints mention only "can be placed on var names (when defined)" and in later section "^long" is an alias for "^java.lang.Long" together gave me an impression I can use it safely in expression like this (def foo ^long 42)

Exactly. The article you linked along with https://clojure.org/reference/vars#metadata I myself also like doing experiments in a REPL and reading Clojure implementation while occasionally stepping through it with a debugger.

got it, thanks for the link, I see it now 🙂

Eastwood might be able to give warnings on such incorrect type hints. I haven't checked in a while.

A section of its docs gives examples of correct and incorrect type hints, but I do not know if it needs updating for more recent versions of Clojure.

Eastwood indeed has some support in its wrong-tag linter Anyway I recommend studying https://clojure.org/reference/java_interop and specifically https://clojure.org/reference/java_interop#optimization the rules are not that hard and then boom, you're an expert

> "^long" is an alias for "^java.lang.Long" this seems wrong to me, primitives and boxed types are entirely different things. clj has support for both

The document does not state that long is an alias for java.lang.Long. Nowhere in the doc does the string java.lang.Long appear. IF you want to have a primitive on a def form you need to use the symbol itself - unliked function arguments or lots of other things:

user> (def ms-time 100)
#'user/ms-time
user> (Thread/sleep ms-time)
Reflection warning, *cider-repl cnuernber/dtype-next:localhost:37165(clj)*:54:7 - call to static method sleep on java.lang.Thread can't be resolved (argument types: unknown).
nil
user> (def ^{:tag 'long} ms-time 100)
#'user/ms-time
user> (Thread/sleep ms-time)
nil 

right, my mistake was that I didn't know that meta tags got resolved for var names unlike for local bindings

> IF you want to have a primitive on a def form you need to use the symbol itself That still won't work - vars can't hold primitives. The tag will be useless. (Or maybe I'm wrong, hold on)

Yeah, I'm wrong - it'll be used, even though the var won't contain a primitive. The tag will be used only to select the right method and won't affect the actual value. Hence you can do stuff like (def ^{:tag 'long} x "x") and later get a ClassCastException when you pass it to some function that expects a long.

What do you mean by "crashing"? Is there an exception, do you have a stacktrace? Chances are, it's not doseq itself but rather some lazy collection that ends up doing more than you'd like.

i'm using kubernetes, all i can see is that in the middle of the job, the REPL connection closes and i can see the kubernetes pod restarted

Then I'd say that the correct first step would be to set up proper monitoring - one where you'd get the error details instead of just the most superficial symptoms.

Or at the very least you could share the code that generates the seq and later consumes it, assuming you can do it given its licensing and there's not too much of it. Maybe you hit an OOM somewhere because you somehow accumulate the results. Maybe it's a stack overflow because you used concat excessively, maybe something else. Impossible to tell.

(let [db-conn (db/_d)]
    (doseq [[u] (vec (user/all-user-ids db-conn))]
      (println u)
      (import-es-user (res/user-to-json (db/by-id u db-conn)))))

So i'm using datomic... user-to-json just serializes the object to JSON

As you can probably guess, nothing's wrong with the above code, except for the fact that you use vec an thus query all the data eagerly. But maybe that's how one's supposed to work with Datomic, I have no clue. So it means that the error is somewhere else. As I said, impossible to tell without some details. Ideally, the stacktrace.

right, well how does one profile where a function might cause a spike in RAM usage

I'd start with VisualVM. But you're assuming that it's an OOM. It might easily be something else.

I would just surround that with a (try (let ...) (catch Exception e (.printStackTrace e))) since you have a println already there I assume you have access to the standard output log of that process

Some things to add to the above: • You should catch Throwable instead • If it's OOM, it might not get printed • You should rethrow the error to make sure things work as intended • Full-on JVM crashes can't be caught (e.g. when there's a bug in the JVM or when you're using some native library that crashes)

I don’t know if it is relevant in your case, but it is worth noting that Clojure lazy seqs easily blow the stack, e.g. (reduce concat (map vector (range 10000)))

Are you sending the http request blocking or non blocking?

Try to also find out why the pod is restarting by looking at the pod events. It may reveal stuff like OOMKilled or probe failures.

My guess is if sending the http requests is not blocking they queue up faster than they can be sent and that could lead to oom

If k8s has 137 as the oomkilled signal maybe you can see that somewhere in your ops system?

A few tips • You can use profilers like VisualVM and YourKit - they are often more heavyweight but can reveal interesting findings like what paths allocate the most memory ◦ see e.g. https://www.yourkit.com/docs/java/help/allocations.jsp • You can Dump the memory on OOM (see https://www.baeldung.com/java-heap-dump-capture). you can then inspect it in some of the profilers (see above) or heap dump analyzers (like Eclipse MAT)

java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=<file-or-dir-path>

Just an aside on this: > I don’t know if it is relevant in your case, but it is worth noting that Clojure lazy seqs easily blow the stack, e.g. (reduce concat (map vector (range 10000))) This is just an issue with the way you are using reduce and concat together to combine seqs together. This works fine:

(apply concat (map vector (range 10000)))

There's also #C0AB48493

https://github.com/hyPiRion/lein-shell should do it

sounds like some other middleware is reading the body before json decoding gets to it

a way to check would be to use reflection to get access to the private pos field (https://github.com/http-kit/http-kit/blob/master/src/java/org/httpkit/BytesInputStream.java) and verify that it indicates the stream has been completely read

Thanks @U0NCTKEV8 thats useful feedback. And i agree, i think some middlware before wrap-json-body has changed, or i didn't configure it properly when we updated compojure-api form 1->2 (which i know regret as it looks like 2 is all "alpha" and wont get support, i should have just worked on switching to retit!)

How would reflection help check if other middleware had read it? i haven't done much with reflection directly.

it is a private field so you can't read it without reflection

https://github.com/arohner/clj-wallhack/blob/master/src/wall/hack.clj has some examples of how you would use reflection to read a private field, or you can just use the library

Another alternative is to add another middleware early in the stack that replaces the input stream with a mock input stream that throws an exception on read.

It won't fix your problem, but it might help identify which middleware is consuming the body

you can probably .reset the body InputStream to read it again

parameter parsing middleware is usually sus

@U7RJTCH6J yep! We added middle ware that captured input and output between the existing mw. The only frustrating part was that when i captured the input on the offending mw, it looked fine in a simple example and in the real example it looked like things had gone sour before it. Oh well. Thanks for the idea.

@UAEH11THP I'll try reset. Thanks.

I understood from what you said that the InputStream was being consumed, but i'm also able to reproduce it in a small example.

(def foo "foo")
  ;; => #'centriq-web.web-server/foo

  (def byte-input-stream (new BytesInputStream (.getBytes foo) (.length foo)))
  ;; => #'centriq-web.web-server/byte-input-stream

  (slurp byte-input-stream)
  ;; => "foo"

  (slurp byte-input-stream)
  ;; => ""

  (slurp (.bytes byte-input-stream))
  ;; => "foo"