This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-07-18
Channels
- # aleph (7)
- # announcements (11)
- # beginners (186)
- # calva (17)
- # cider (26)
- # clj-kondo (4)
- # cljdoc (12)
- # cljs-dev (3)
- # clojars (1)
- # clojure (105)
- # clojure-berlin (1)
- # clojure-chicago (1)
- # clojure-dev (34)
- # clojure-europe (3)
- # clojure-italy (4)
- # clojure-nl (27)
- # clojure-russia (19)
- # clojure-uk (25)
- # clojuredesign-podcast (4)
- # clojurescript (54)
- # cursive (6)
- # data-science (1)
- # datascript (11)
- # datomic (5)
- # emacs (3)
- # events (2)
- # fulcro (13)
- # graalvm (5)
- # jobs (15)
- # leiningen (7)
- # luminus (3)
- # melbourne (1)
- # nrepl (1)
- # nyc (2)
- # onyx (4)
- # pathom (6)
- # pedestal (18)
- # re-frame (19)
- # reagent (10)
- # shadow-cljs (27)
- # spacemacs (32)
- # sql (11)
- # tools-deps (35)
- # vim (50)
@seancorfield I updated my CI to openjdk-11 from openjdk-8 and my uberjar step using depstar has started failing with some sort of memory issue. The uberjar is built with clojure -J-Xmx3g -A:uberjar -v
and the command exits with Killed
and the status code 137, which typically indicates a memory issue. I'm using the latest version 0.2.4. Any thoughts on why depstar would have a memory issue on Java 11?
I am using the -v
option for depstar. It's interesting that the build always hangs on this file /home/circleci/.m2/repository/http-kit/http-kit/2.3.0/http-kit-2.3.0.jar
for a few seconds right before the Killed
message.
It's happening on CI and according to CircleCI that usually means a mem issue: https://circleci.com/docs/2.0/java-oom/#debugging-java-oom-errors
It works with 0.1.7:
clojure -J-Xmx3g -Sdeps '{:deps {seancorfield/depstar {:mvn/version "0.1.7"}}}' -m hf.depstar.uberjar MyProject.jar
The code of depstar is small, should be fairly easy to find the issue. Must have something to do with Java 8->11.
This doesn't sound relevant but I found this https://bugs.openjdk.java.net/browse/JDK-8222532 which is resolved in Java 11.0.4. I'm running 11.0.3.
It's not obvious how to update to 11.0.4 for some reason. Maybe the debian packages haven't updated or something.
It's possible that issue is relevant but it's proving hard to validate given 11.0.4 isn't out yet. That issue may not even be the problem. Going to roll back to Java 8 for now.
There doesn't appear to be a debian package yet. Also couldn't find a command to force update to 11.0.4. All guides seem to suggest using an install path that doesn't let you select your version. I don't see a 11.0.4 version on their docker page yet: https://hub.docker.com/r/adoptopenjdk/openjdk11
I see 11.0.4 here though: https://adoptopenjdk.net/releases.html?variant=openjdk11&jvmVariant=hotspot
The change with depstar
was from expanding everything out to a temporary folder and then copying it all back into a ZIP (JAR) to just copying everything into a ZIP (JAR) directly -- which is why 0.1.7 is so much slower than 0.2.x. It now uses a zip filesystem directly. I wasn't aware that changed the memory usage but it's certainly possible. All our uberjars are 30-40MB each and we haven't seen any memory issues using the default heap settings running clj
...
It definitely has to do with java 11. I reverted our build and prod images to java 8 and everything builds as expected - no mem issue.
The change also served as a solution for Windows-related issues with reserved filenames (so you couldn't run depstar
0.1.7 at all on Windows when you depended on libraries that contained certain filenames!).
Good to know. We're on Adopt OpenJDK8 everywhere because some stuff we rely on won't run on JDK11.
We've been using CircleCI's pre-built images for building uberjars. Specifically circleci/clojure:openjdk-11-tools-deps-1.10.0.442
is the one that produces the mem issue. Not sure if they use adopt openjdk or not.
@U083D6HK9 we use the latest versions and the project works on openjdk 8,9,11,12:
http-kit {:mvn/version "2.4.0-alpha4"} ;; fixes java 11
javax.xml.bind/jaxb-api {: mvn / version "2.3.1"} ;; support java 9+
org.clojure/core.rrb-vector {: mvn / version "0.0.14"} ;; support java 11+
see more info about http-kit: https://github.com/http-kit/http-kit/releases/tag/v2.4.0-alpha3@U1EQNSHL4 Not sure I understand what you mean. This issue was about building an uberjar with depstar, not an issue with http-kit.