Fork me on GitHub
#leiningen
<
2019-10-27
>
zlrth16:10:14

I’m calling a function from the commandline like so:

JVM_OPTS="-Xmx8G" lein run -m myproject/myfunction
it’s out-of-memory’ing, no matter what i set Xmx to. looking a little more, I see via jps -lv that the process being called looks like:
26548 clojure.main -Dfile.encoding=UTF-8 -Xmx8G -Xmx4g -Dclojure.compile.path=/home/app/myproject/target/classes -Dcollage.version=0.1.0-SNAPSHOT -Dclojure.debug=false
note the two Xmx flags. my java program is taking the second Xmx value. (i get the same results as this SO commenter[0]) the -Xmx4g comes from my project.clj (or at least, that’s only place Xmx4g appears i can find). looking at the tutorial, I’d say that JVM_OPTS should overwrite a project.clj’s :jvm-opts[1]. [0] https://stackoverflow.com/a/26727332/3925569 [1] https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md#setting-jvm-options

zlrth16:10:29

this open github issue[0] marked for the 3.0.0 release speaks to my behavior. this indicates to me that i should not expect an environment variable will override project configuration. ok. how do i set an -Xmx on a per-invocation basis? [0] https://github.com/technomancy/leiningen/issues/1680

mikerod19:10:41

@mfm per-invocation: how about

lein update-in : assoc :jvm-opts '["-Xmx8G"]' -- run -m myproject/myfunction

mikerod19:10:15

alternatively, you could make a :profiles entry that set :jvm-opts when a certain profile was used via with-profile

mikerod19:10:26

you could also add an :aliases entry to do something like above if you knew it was a common case

mikerod19:10:18

you could also avoid having the project have a default :jvm-opts by moving the project’s own :jvm-opts (of 4g as you say above), to a profile, and only have that profile active via a certain :aliases so that it doesn’t interfere with someone just wanting to use JVM_OPTS w/ lein run

mikerod19:10:27

I typo’ed my update-in above too, will fix - fixed

zlrth19:10:14

ah--thanks for lein update-in. trying that now. should have added this context: in production and staging, we happen to use lein run -m myproject/myfunction for basically scripts. so i’d like to have per-invocation setting of xmx for those calls, because the amount of ram they should have differs, both from each other and from project.clj

zlrth19:10:33

(not to correct anything you’re saying; what you’re saying is compatible)

mikerod19:10:47

yeah, makes sense

mikerod19:10:57

update-in may be what you want

mikerod20:10:08

there may be an issue of profile merging still, if so, have to probably make it uglier

mikerod20:10:24

lein update-in : assoc :jvm-opts '^:replace ["-Xmx8G"]' -- run -m myproject/myfunction

mikerod20:10:58

or probably better yet, in your project.clj where you have :jvm-opts, do ^:displace metadata instead. I may be wrong, can’t remember. update-in may not try to merge when you do assoc (now that I think of it, I don’t recall ever seeing a merging issue like I’m bringing up here)

zlrth20:10:20

thanks! i can try out some different things

mikerod20:10:05

yep, let me know if things don’t work out

zlrth20:10:10

sanity check: am i correct that the tutorial is misleading? > You can also pass options to Leiningen in the JVM_OPTS environment variable. https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md#setting-jvm-options

mikerod20:10:21

you can also add DEBUG=true lein ... to sometimes get some more verbose details of what is happening.

zlrth20:10:27

oh nice--thanks

mikerod20:10:24

The line > You can also pass options to Leiningen in the JVM_OPTS environment variable. is perhaps just too vague if I understand the problem correctly - that JVM_OPTS takes “lower priority” than project :jvm-opts

mikerod20:10:53

I assume that’s what you are referring to by “misleading” right?

mikerod20:10:10

I think JVM_OPTS do work when you don’t have :jvm-opts though.

zlrth20:10:32

yeah--i actually meant “wrong” but didn’t want to say it, but i was wrong for thinking it’s wrong per se.

zlrth20:10:11

i now see that that they don’t specify priority

mikerod20:10:00

Yeah, I looked briefly at the source - not enough to fully understand the end result, but in leiningen.core.eval/get-jvm-args

mikerod20:10:29

so it reads both there, how that works out beyond there is a bit unclear still

mikerod20:10:53

if you use DEBUG=true may be clearer. I almost wonder if we end up with like java -Xmx8g ... -Xmx4g ... and that just is JVM dependent

mikerod20:10:49

think they are stored in some sort of hash-map in JVM typically, wher the 2nd one read tends to “win”

mikerod20:10:06

but at this point - just hypothesizing, so will quit while ahead 😵

zlrth20:10:42

thank you very much!

zlrth20:10:18

this:

lein update-in : assoc :jvm-opts '^:replace ["-Xmx8G"]' -- run -m myproject/myfunction
works for me (for anyone reading, i added a - in front of Xmx)

zlrth20:10:54

“works” means “the output of jps looks right” and i’ll post again in celebration if the function doesn’t OOM

zlrth20:10:56

thanks again!

mikerod20:10:25

ah dang - when I edited didn’t have the lead - sorry

zlrth20:10:36

no problem!