? ? ?

the sound of techno

In the #beginners channel? Perhaps more appropriate for #music or #off-topic ?

Yes, we are using JDK 8. And the errors all occur at the same time. All API calls seem to fail. Most request handlers just barf. The CPU spikes. Locally I added the -XX:+*HeapDumpOnOutOfMemoryError* and set the Metaspace to a low value so I can replicated the issue. Now I am using the heap analyzer tools but honestly I can barely make any sense out of it. I see references to some of our code but I can't really make sense of the big picture.

Note that even a simple clojure application demands at least 30-40 MB - depends on how much code & dependencies you load. If your app consumed gigabytes then that might be a real problem - would be good to have a continuous monitoring of various aspects of the JVM memory consumption; it's not super-specific to your problem but I always recommend reading this SO question to get a better understanding of JVM memory consumption: https://stackoverflow.com/questions/53451103/java-using-much-more-memory-than-heap-size-or-size-correctly-docker-memory-limi/53624438#53624438

And here some nitty-gritty details about Metaspace: https://stuefe.de/posts/metaspace/sizing-metaspace/

A closure only retains values bound to its free variables. Both sequences are candidate for GC when foo returns.

There's also an interesting version of fn* using :once true metadata for one-shot closures: https://stackoverflow.com/questions/23586710/what-is-fn-and-how-does-it-differ-from-fn

@U053XQP4S, so closures contain only the values bound to free variables, not to the environment frames which contain those variables? If that's so, that's terrific.

I don't think this behavior is documented anywhere but that's what the compiler actually does and it's very reasonable to rely on that IMO

I wonder whether excessive use of lazy sequences might be the culprit. A raw sequence only contains final values, even if very many of them. However, a lazy sequence must contain references to all values necessary to compute the rest of the sequence. It could be that in extreme cases lazy sequences occupy much more memory than the sequences themselves.

(defn foo [x]
  (let [very-long-sequence (compute-very-long-sequence-of-sequences x)]
    (map count very-long-sequence)))

right

so as I understand, I can wrap all generations of such lazy sequences with doall for force them to be non-lazy. But finding all such lazy generations may be challenging. Not really sure how to do that.

or use eager constructions like vectors, transducers, reductions

sorry, don't know how to do that. And it sounds like a difficult and invasive refactoring of my codebase, especially since it is not guaranteed to solve the problem.

then doall sounds good

map is documented as returning a lazy sequence. but it is not clear whether mapcat returns a lazy sequence or not.

after a bit of testing, it is promising but not conclusive. I get far fewer out of memory errors after forcing all calls to map and mapcat to be non-lazy. My tests now run in about 2 minutes, whereas they previously took a half hour to run, But occasionally still give out-of-memory errors. But less often.

very interesting results.

AND the cpu meter I'm using on my Mac was showing huge stress, now is much happier and the cooling fan is much quieter.

Is this the sample app from the Heroku guide or your own reagent app?

I essentially copied everything over

The sample reagent app deploys correctly, btw.

My only thought is how bootstrap is being pulled in. I use a CDN for CSS libraries. If using local libraries then check the path to them is correct when deployed on Heroku

Posting a link to the logs or adding them as a thread would be a preferred way to share, avoiding taking up lots of space in the main thread. Thanks

Not really. In fact it’d be great if I can just copy paste my public folder and let it serve my files

The reason why I think I need a backend is because I need to store input from users. I’ll need a database for this, right? My impression is that you can’t do this with Netlify and other static hosts.

(Sorry, I’m really new to web dev but otherwise not that new to programming in general. Hope you understand.)

clj
Clojure 1.10.1
user=> (require 'clojure.set)
nil
user=> (in-ns 'clojure.set)
#object[clojure.lang.Namespace 0x5a772895 "clojure.set"]
clojure.set=> (defn union [x y] (prn "i've changed this"))
#'clojure.set/union
clojure.set=> (in-ns 'user)
#object[clojure.lang.Namespace 0x1c12f3ee "user"]
user=> (clojure.set/union #{:a} #{:b})
"i've changed this"
nil
user=>

(ns my.ns
   (:require [clojure.set])
(in-ns 'clojure.set)
(defn union [x y] (prn "i've changed this"))
(in-ns 'my.ns)
(clojure.set/union #{:a} #{:b})

can you show the stack trace?

Yeah hang on

But when I move it into a function (e.g. wrap the above lines inside a function) and then evaluate that function at the REPL, it works

So it would seem that union is being called elsewhere after I redefine it, when the reader goes through?

Would that make sense?

what does the stack trace say?

it should tell you where its called with three args. and union has a much more rich signature than what we did just for example purposes

([] [s1] [s1 s2] [s1 s2 & sets])

Sorry I have to run off but I appreciate the help and will let you know if I get to the bottom of it :)

If you're stuck on Windows 7, you probably can't install a recent enough Powershell to be able to use the new Clojure CLI.

But ask in #clj-on-windows in case folks there know more about Windows 7 and Powershell than I do (very likely -- I'm a lot more familiar with Windows 10).

Thanks. Hmmm upgrade to Windows 10 with the new laptop will be next year. I guess lein.bat is sufficient for now to learn clojure basics.

A lot of the tutorials out there still use Leiningen as well as many of the books, so you should be fine with lein for learning. Windows is a bit of a second-class citizen in the Clojure world, so even with Windows 10, it's a lot easier to do everything using WSL2 which is how I work with Clojure: WSL2 with Ubuntu, Clojure CLI, use VcXsrv (Xlaunch) on Windows to support X11 apps, then run everything on WSL2, including my editor.

The Clojure code itself will be the same regardless of if using Leiningen or Clojure CLI tools. The difference is is the details of how you configure Clojure projects and even then there is much cross-over.

Thanks Sean and John.

John, i found your Youtube channel just now. I subscribed to it. Watching 4clojure guides playlist now.

Feel free to ask any questions about my content in the #practicalli channel or send me a direct message.

It comes from DB. It returns things like:

{:name "Maria"
 :kids []}