This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2015-10-22
Channels
- # admin-announcements (29)
- # aws (2)
- # beginners (25)
- # boot (110)
- # business (15)
- # cider (39)
- # cljs-dev (3)
- # clojure (90)
- # clojure-czech (28)
- # clojure-hamburg (1)
- # clojure-japan (24)
- # clojure-poland (149)
- # clojure-russia (46)
- # clojure-sg (9)
- # clojure-uk (6)
- # clojure-ukraine (1)
- # clojurescript (105)
- # core-async (37)
- # cursive (9)
- # dato (7)
- # datomic (6)
- # emacs (10)
- # events (1)
- # hoplon (22)
- # jobs (4)
- # ldnclj (38)
- # leiningen (4)
- # off-topic (17)
- # om (173)
- # onyx (134)
- # re-frame (46)
- # reagent (35)
I wonder if we could take some inspiration from tools like gulp for the above problem.
Hi, so if you want to shell out from a boot task, you'd use the dosh
function in boot.util
, right? But when the command needs sudo privilege, is there going to be a prompt? I haven't tried it yet. Wondering out loud.
I'm not explaining myself well, I'll do some tests and rephrase my question. Please ignore for the moment.
OK, so my question is, is it possible in boot to shell out interactive commands, ie. commands that interact with the user, prompting for input, like a sudo
prefixed command.
@danielsz: dosh doesn't handle interactive commands, it was made to stream output from the process as it's produced rather than after the process has exited
@micha: I have two things where I wanted to ask you for your advice: 1. How would you go about debugging a memory leak in Boot (specifically pod-pools) 2. In general I’d like to get a better understanding of where time is spent in Boot — just use something like yourkit?
for 1. i used visualvm in the past and was able to track down issues with pods pretty well
for 2 yourkit is probably the thing to use, we could maybe get a sponsored license for boot
it’s free for OSS (I think)
I tried using Visualvm some time ago and it was SUPER SLOW in compared to YourKit
It had to instrument each class and with cider-nrepl and etc. there's like 50k classes in classpath. With some filters I managed to get it to something like 5000 classes and then it was only 5 minutes instead of 30 minutes to start CPU profiling.
yourkit seems pretty snappy
@micha: when using make install
can I be certain that stuff is rebuilt?
@martinklepsch: you can do make clean
first
Getting No such task (./pod-loop)
when just running a script file similar to this one: https://github.com/alda-lang/alda/blob/master/bin/alda
even when using the very same file…
./pod-loop
also tried adding the dir to PATH and running without ./
I also tried with the contents of the alda script linked above
(.clj was appended by emacs)
empty dir
lib 2.3 app 2.2
should probably try with app 2.3 although that hasn’t caused any trouble until now
deleted that, same
hm. now it’s getting weird.
wow. it was app 2.2
works fine with 2.3
I’ll check again if that really was the difference or if I maybe did something else too which I forgot about
@martinklepsch: by any chance do you use windows? i've had reports from windows users trying to run the alda
boot script of a similar error message
can I invoke a file using boot similar to how it’s done when using #!
?
@martinklepsch: how do you mean?
boot —file pod-loop
@micha: do you have 10min to chat?
sure, will dump some thoughts here
I’ve been looking into some of the memory issues that were reported with boot-garden (https://github.com/martinklepsch/boot-garden/issues/10) and also with boot in general (https://github.com/boot-clj/boot/issues/268). While all of these cause OutOfMemory
exceptions it seems that they’re solely related to PermGen space or “Metaspace" in Java 8.
While increasing PermGen size fixed the issue for Java 7 the new behavior in Java 8 is to have unlimited “Metaspace”. So if there are lots of classes generated this space can fill up your entire RAM & crash the process. With PermGen this problem surfaced earlier but still should have ultimately caused OutOfMemory
exceptions.
Now reading up on when Metaspace things are GC’d it seems that it’s tricky and “subtle references may leak and prevent the Classloader from being reclaimed” (http://stackoverflow.com/questions/2344964/when-and-how-is-a-java-classloader-marked-for-garbage-collection).
There are some potential workarounds mentioned in the above SO post but before I dig deeper I’d make sure I’m not running in the wrong direction full speed. Does all this sound reasonable to you?
The setting for it is gone but there’s still a special place for class definitions, now called Metaspace
using -XX:MaxMetaspaceSize=200m you can make the occur faster: https://gist.github.com/martinklepsch/ac4c4680e3f47c5c2df9
I found a few things that are not being closed properly along the way ^
I’m not sure if this is correct but probably someone else can review (it works ;P)
no it’s not
@micha: right, should have checked that earlier, will give it a try with yourkit and ping back
was just going to say that
Is it reasonable to try boot-clj for a project that essentially requires sub-projects packaging like lein-sub does?
@jfntn: I’m not doing it myself but would say yes, see https://github.com/decomplect/ion/blob/master/build.boot for an example
@martinklepsch: oh thanks that looks just like what I was going for!
I don’t see any inter-project dependencies though, would you just be able to add a deps entry for previously built artifacts and make sure that the pipeline builds them in topological order?
@micha: if you scroll to the “AnonymousClassLoader” section that sounds relevant: http://blog.headius.com/2008/09/first-taste-of-invokedynamic.html
@jfntn: you mean like you have subprojects A, B & C and A & B depend on C so C should be built first when building all?
as long as you don’t want to AOT or uberjar A or B this shouldn’t be an issue at all I think
if you want to do that you’ll have to take care of it yourself
Right, but in theory you should be able to uberjar C first, and make sure the artifact ends up on the classpath for A & B when you uberjar those down the pipeline?
@jfntn: if A or B depend on C you probably don’t want to uberjar C but just use regular resolution via maven. In this case you can just specify C as a regular dependency
I understand that as “a Clojure var holds a reference to the CL” but I don’t see where that would happen
this seems related http://dev.clojure.org/jira/browse/CLJ-1125 /ping @tcrawley
Good night
@martinklepsch, @micha re: mem core.async might be a culprit, https://github.com/projectodd/shimdandy#preventing-memory-leaks