This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-05-06
Channels
- # announcements (5)
- # aws (28)
- # babashka (4)
- # beginners (163)
- # bristol-clojurians (2)
- # calva (4)
- # cider (18)
- # clj-kondo (30)
- # cljs-dev (28)
- # cljsrn (50)
- # clojure (96)
- # clojure-europe (25)
- # clojure-italy (6)
- # clojure-losangeles (1)
- # clojure-nl (4)
- # clojure-sweden (7)
- # clojure-uk (32)
- # clojurescript (39)
- # conjure (74)
- # cursive (12)
- # events (1)
- # fulcro (32)
- # helix (71)
- # jackdaw (2)
- # leiningen (10)
- # off-topic (14)
- # pathom (59)
- # rdf (7)
- # re-frame (6)
- # reitit (28)
- # ring (7)
- # shadow-cljs (207)
- # slack-help (2)
- # spacemacs (3)
- # specter (7)
- # sql (12)
- # tools-deps (14)
- # xtdb (32)
Hi! I'm trying to debug a weird memory issue when running clojure inside Docker containers. I have multiple clojure services (each on a different container, JVM, and so on), set up with sensible memory limits (i.e. ~600MB per process). The thing is, I've recently upgraded my dev machine to 32GB of RAM. And code that worked just fine previously is now crashing with OOM errors! What I mean is that the JVM inside the container reaches its cgroups memory limit, and docker kills it, not that I'm seeing OutOfMemoryError exceptions. I'm using OpenJDK13 and the container-related flags are properly set up using deps.edn:
{:jvm-opts ["-XX:+UseContainerSupport"
"-XX:InitialRAMPercentage=20"
"-XX:MaxRAMPercentage=80"]}
What could be causing this? Any ideas? :Sare you 100% certain the flags are being actually applied? ps aux
can be a way to check it, JMX being a more hardcore way
@U45T93RA6 I thought I'd "made sure" by checking the JVM args from whithin a running repl (with this: https://stackoverflow.com/a/5317220). The flags were there. Could it happen that the flags are ignored despite being parsed by the JVM?
in ps aux | grep java
you an see all the flags/args that were passed to the final Java invocation running your program
Aren't you running multiple JVM instances (one per service), each with "-XX:MaxRAMPercentage=80"
? That'd surpass 100% for n>=2
@U45T93RA6 yes, that's what I do, but if you set up a memory limit with docker (uses cgroups internally) the JVM will use that limit and treat it as the total RAM, that's why you can set such a high value
and it seems to work as advertised (at least in some cases?). If any service tried to take an initial portion of 16GB (50%) of the system memory, docker would immediately kill it, but that's not what's happening :S
got it. you might have luck by asking this in stackoverflow as this can be posted as a generic jvm question
Is there an example of running clojure using tools-deps from the Clojure image on Docker? https://hub.docker.com/_/clojure
I'm not aware of any example, but it's not very different from running it locally. If you get the tools-deps version of the clojure docker image, you basically get a system with the clj
command.
Yea I just noticed
I guess a related question is. How does one ensure dependencies are only installed once?
the first one will work on any docker version, you need a dockerfile that pretty much does:
1. Copy the deps.edn file (only! not the sources)
2. Run clj
, which will fetch the deps declared in deps.edn
3. Copy the source code
4. Builds the project (e.g. uberjar with depstar), runs a REPL, or whatever you need
This is what I have so far
FROM clojure:tools-deps-alpine
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
RUN clojure -A:depstar -m hf.depstar.uberjar MyProject.jar
COPY . /usr/src/app
CMD java -cp MyProject.jar clojure.main -m core
But I get an error.
Step 1/6 : FROM clojure:tools-deps-alpine
---> b8908541da81
Step 2/6 : RUN mkdir -p /usr/src/app
---> Running in 0fbcfb5eee31
Removing intermediate container 0fbcfb5eee31
---> 91928ecb9d2b
Step 3/6 : WORKDIR /usr/src/app
---> Running in 67e70492e8c1
Removing intermediate container 67e70492e8c1
---> f087ebf44efd
Step 4/6 : RUN clojure -A:depstar -m hf.depstar.uberjar MyProject.jar
---> Running in 9417b55e83cd
Error building classpath. Specified aliases are undeclared: [:depstar]
With this first method, you create a base image with the dependencies, and that image only needs to be rebuilt if you change your deps.edn file, because it's the only thing you copied. Then changes in the sources will not trigger to redownload the dependencies
Hmmm.. from what I can see, your dockerfile never copies a deps.edn file. Even if you have a ~/.clojure/deps.edn
in your home folder, that won't be available inside the container, so the depstar alias will not be defined
Is it too much to ask for the script for the Dockerfile? 😅 I’m new to this Docker stuff
I’m trying :thinking_face:
FROM clojure:tools-deps-alpine
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY deps.edn /usr/src/app/
COPY src/ /usr/src/app/
RUN clojure -A:depstar -m hf.depstar.uberjar MyProject.jar
COPY . /usr/src/app
CMD java -cp MyProject.jar clojure.main -m core
Something like this. But I can't check this works right now. Our actual Dockerfile is a bit more complex than that and I had to strip it down to a simple version:
FROM clojure:openjdk-13-tools-deps-1.10.1.483-buster AS base
WORKDIR /usr/src/app
COPY deps.edn .
# Download all dependencies
RUN clojure -A:uberjar-deps -A:cider-deps -e '(println "Downloaded all dependencies")'
# Copy the source code
COPY src src
COPY test test
# AOT-compile
RUN mkdir classes
RUN clojure -A:cider-deps -e "(compile 'my.main.namespace)"
# Generate uberjar
# NOTE: It is very important to add any aliases that contain build dependencies in the --aliases flag, not before
RUN clojure -A:uberjar-deps -A:uberjar \
--main-class my.main.namespace \
FROM openjdk-13-slim-buster as prod
WORKDIR /usr/src/app
COPY --from=builder /usr/src/app/target/app.jar .
CMD ["java", "-XX:InitialRAMPercentage=50", "-XX:MaxRAMPercentage=80", "-jar", "app.jar"]
Thanks. Will take me some time to reverse engineer what is going on 🙂
Yes, it took us a while to get to this 😅. The real one is actually 3x longer (but none of it is relevant to the basic use case). Note that this assumes:
• A :cider-deps
alias that declares dependencies for cider/nrepl (that's not necessary if you don't need to run a REPL inside the container)
• A :uberjar-deps
dependency that declares the dependencies for depstar, to avoid redownloading for every build
• A :uberjar
profile that sets the right main class to point to depstar.
I've taken extra care to avoid redownloading dependencies as much as possible, but if you really need to avoid redownloading dependencies in a more fine-grained way you'll need to use buildkit, and particularly what's called a buildkit build cache. It's all "experimental" for now, so if you don't build that often and are fine with redownloading dependencies every time your deps.edn file changes, I recommend this approach
Also relevant: There are many gotchas when it comes to running a JVM inside docker (see my previous message). Google that for more info. This Dockerfile script already takes care of the basics by: • Using a recent JDK version, with container support enabled by default • Specifying an Initial and Maximum ram percentage You should also make sure to limit the memory of your container with docker. Although I'm having issues with that right now (see my previous message). So take all this with a grain of salt 😅. It's mostly been working so far.
So :uberjar-deps
is for the dependencies I need in production.
Can you please explain the :uberjar
profile a little more?
That’s fine haha. I really appreciate your input! @U70027S0N Very little people use Docker containers with Clojure so the information available online for it is slimmer than normal haha
@U05095F2K yeah, I think it makes more sense to just show you the profiles:
:uberjar-deps {:extra-deps {uberdeps {:mvn/version "0.1.8"}}}
:uberjar {:main-opts ["-m" "uberdeps.uberjar"]}
so basically I split uberdeps into two profiles, that's the trick I mentioned to avoid redownloading uberdeps for every build
Yeah, It's weird how very few people have tried this. Maybe it's just that people are doing it but not talking about it? :man-shrugging:
Maybe it’s so painful and each setup seems unique to their use case that the pain of simplifying it and putting it into a blog post is too much?
Yeah, perhaps.. It's probably my case as well. I like sharing knowledge but I don't really have time to put all this knowledge in a public blog
Everything works well for me except for this.
Step 8/12 : RUN clojure -A:uberjar-deps -A:uberjar --main-class app.core
---> Running in 9363d035db23
[uberdeps] Packaging target/app.jar...
+ classes/**
+ src/**
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
+ org.clojure/clojure 1.10.1
. org.clojure/core.specs.alpha 0.2.44
. org.clojure/spec.alpha 0.2.176
[uberdeps] Packaged target/app.jar in 712 ms
Removing intermediate container 9363d035db23
---> e272737d87e2
Step 9/12 : FROM openjdk:14-slim-buster as prod
14-slim-buster: Pulling from library/openjdk
54fec2fa59d0: Pull complete
b7dd01647a92: Pull complete
e4cd17a79ec4: Pull complete
10aaecc16bec: Pull complete
Digest: sha256:4190b9aa1ba25ee0082234db30930bcb37ba3eafafbbb937986be24528d8e6ec
Status: Downloaded newer image for openjdk:14-slim-buster
---> 3b9a9e451fca
Step 10/12 : WORKDIR /usr/src/app
---> Running in bf3fa1b8dbaa
Removing intermediate container bf3fa1b8dbaa
---> e63bd7f4f079
Step 11/12 : COPY --from=builder /usr/src/app/target/app.jar .
invalid from flag value builder: pull access denied for builder, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
I got it working. Ended up reverting to depstar
the background is I'm using clojure with spark, and try to let the workers execute commands like (defn foo [x] (* x x))
(this gets foo
defined on the worker), so later I can use (spark/map rdd foo)
nailed it
(defn spark-eval [ns form]
(let [evaled-form `(binding [*ns* (find-ns (quote ~ns))]
(eval (quote ~form)))]
(println evaled-form)
(->> (spark/parallelize @_sc (repeat 1 evaled-form))
(spark/map eval)
spark/collect)))
(defmacro defnk [name args & body]
(let [fdef `(defn ~name ~args ~@body)
ns (symbol (str *ns*))]
(spark-eval ns fdef)
fdef))
why (repeat 1 evaled-form)
rather than [evaled-form]
?
or I guess (list evaled-form)
if you need lists to get the right behavior
one thing that tricks me is this part
`(binding [*ns* (find-ns (quote ~ns))]
(eval (quote ~form)))]
for me it shall be the same with this one because eval and quote cancels each other
`(binding [*ns* (find-ns (quote ~ns))]
~form)]
but I find the latter does not work - the ns in effect would still be clojure.core
you could also use something like (intern (quote
ns) (quote sym) (eval (quote ~form)))` if you limited yourself to def and passed a symbol to bind to
yesterday I discovered the clojure.test/assert-expr
multimethod - it introduces a new syntax that works inside is
is any clojure ide / editor integration smart enough that it could find the implementation of a syntax introduced via this sort of indirect extension without hard-coding each known usage of the pattern?
that is, fireplace couldn't find it via [d
or [C-d
, but I wonder if any environment would be able to do that lookup?
@pppaul sure, but how would I even get from (is (in? x y))
to (defmethod clojure.test/assert-expr 'in? ...)
in the defining ns?
without special casing the fact that the is macro uses a multimethod over symbols to create assertions
why would I be looking for assert-expr when what I see is (in? x y)
clojure.test/is seems like a fairly special flower to me in this regard. I wrote special handling code for it in Eastwood years ago, I know, and was somewhat surprised how special of a flower it truly is.
haha, yeah I understand that now... definitely not a pattern I want to use in my own code or extend...
I can, how would I get from is to assert-expr without special casing is?
I suspect I can't and this is a limitation of this sort of extension style, but maybe I'm missing something
yeah, somewhere down the stack assert-expr is used, but it's not easy to find / understand
so I think I should avoid this style of exension, as it obfuscates things
personally, I've found the is multi method to be less useful that making macros that produce is statements
Hello folks! Anyone experienced with transit? I would like to get it to send change floats to bigdecimals before sending them (the JS side handles big decimals just fine but floats are displayed as [TaggedValue: f, <value>]
which is not optimal. Thanks!
transit write takes an optional argument mapping types to writers, and the reader takes an optional argument mapping tags to construction functions
@holyjak I have examples of both here https://github.com/noisesmith/poirot/blob/master/src/org/noisesmith/poirot/handlers.cljc and their usage here for writing https://github.com/noisesmith/poirot/blob/master/src/org/noisesmith/poirot.cljc#L83 and here for reading https://github.com/noisesmith/poirot/blob/master/src/org/noisesmith/poirot.cljc#L107
I see it also takes this option: :transform - a function of one argument that will transform values before they are written.
I’m trying to build a jar with uberdeps and getting stuff here when I try and run it.
Exception in thread "main" java.lang.NoClassDefFoundError: clojure/lang/Var
at app.core.<clinit>(Unknown Source)
Caused by: java.lang.ClassNotFoundException: clojure.lang.Var
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 1 more
some info you will probably need to provide for anyone to help - how did you build it? how are your running it? what's in the jar?
Fair point. deps.edn
{:deps {}
:paths ["src" "classes"]
:aliases {:uberjar-deps {:extra-deps {uberdeps {:mvn/version "0.1.8"}}}
:uberjar {:main-opts ["-m" "uberdeps.uberjar"]}}}
src/app/core.clj
(ns app.core
(:gen-class))
(defn -main [& args]
(println "Hello world"))
command line
clojure -A:uberjar-deps -e '(println "Downloaded all dependencies")'
mkdir classes
clojure -e "(compile 'app.core)"
clojure -A:uberjar-deps -A:uberjar --main-class app.core
java -jar target/my-project.jar
if you jar -tf target/my-project.jar | grep Var
do you see anything?
Ah I seem to have fixed it. Needed Clojure as an explicit dependency
I only realised after pasting in the code blocks lol
it should be a dependency already via the install deps.edn
so that implies to me that uberdeps is not doing the right thing
Okay.. yea I usually ignore it.
I looked at the code and it's ignoring the install deps.edn and user deps.edn, so that's a bug in uberdeps imo
Install: agree. User: disagree. Building an uberjar is production-sensitive and should ignore the user deps.edn.
well, that's uberdeps' decision to make. it's currently manuallly merging a subset of install deps.edn
Yet another case where a tools.deps.alpha
-based tool doesn't follow the "expected" behavior regarding the install/user/project deps.edn
files 😐 It's why several of those tools drive me crazy!
I decided to go with depstar. Managed to get it working with a docker image build. Though I am confused with this warning? What does it mean?
Step 6/11 : RUN clj -A:depstar -m hf.depstar.uberjar MyProject.jar
---> Running in 394071736dfb
Building uber jar: MyProject.jar
{:warning "clashing jar item", :path "about.html", :strategy :noop}
{:warning "clashing jar item", :path "about.html", :strategy :noop}
{:warning "clashing jar item", :path "about.html", :strategy :noop}
@U05095F2K It means exactly what it says: It found about.html
in multiple artifacts as it was building the JAR and it ignores the duplicates (`:noop`).
If you want more detail on exactly what files it is looking at, you'll need to specify the verbose flag, either -vv
or -vvv
depending on the level of detail you need. That should show you where it is finding multiple about.html
files.
The weird thing is this is just a hello world print. Nothing more.
Like I said, run depstar
with -vvv
and see where those files are coming from.
RUN clj -A:depstar -m hf.depstar.uberjar MyProject.jar -vvv
should do it.
BTW, when you're running the CLI in non-interactive mode, you probably want clojure
instead of clj
.
They should not directly read deps.edn
and manipulate it! They should rely on t.d.a. and respect the normal merge order and aliases and so on.