ragge: thanks for looking into this!

ragge: interesting, it could be this: http://stackoverflow.com/a/3014860

who would expect it to be opening a stream and not closing it?

@micha: yeah, I thought it might be that too at first... clojure has the same bug: http://dev.clojure.org/jira/browse/CLJ-1659

I tried this: https://github.com/boot-clj/boot/blob/814d9b6ff991c113f323ab2c35a04750a825d0b0/boot/pod/src/boot/pod.clj#L358

current approach is def leaking, but it wasn't what is causing the issue I'm seeing

I also tried fixing another unclosed resource in jar-entries*: https://github.com/boot-clj/boot/blob/814d9b6ff991c113f323ab2c35a04750a825d0b0/boot/pod/src/boot/pod.clj#L301

currently, the JarFile opened isn't closed: https://github.com/boot-clj/boot/blob/76980e6b132f46a8b00cecd5fd2713343cc21a49/boot/pod/src/boot/pod.clj#L301

ugh, disregaqrd those links

I tried these changes: https://github.com/ragnard/boot/commit/814d9b6ff991c113f323ab2c35a04750a825d0b0

however, although I think both are leaking file handles, they didn't cause my problem

I ended up re-writing unpack-jarto use JarFile and control resources more explicitly

https://github.com/ragnard/boot/commit/71786cc4f989241c9bd4d959dcca51bd74b2a5c9

that's fixes my problem

could probably be tidied up a bit, but I had to go to bed

let me know if youre interested in a PR for this

I think, while using URLs is pretty, it comes with a bunch of issues in java stdlib

@micha: @juhoteperi related to pod dependencies I was wondering if it would make sense to add something like the output of boot show -d for each pod when called with -vv — thoughts?

@ragge: excellent sleuthing!

Hi I have some problems with converting an leiningen build to boot

I am currently using trapperkeeper, and trapperkeeper apparently expects your src content so be copied in the jar

So I get the follwing exception after I created a uberjar

java.io.FileNotFoundException: Could not locate nl/avisi/xodus/service__init.class or nl/avisi/xodus/service.clj on classpath.

When I inspect the Jar I see this is true? Is there an option to force this?

@ragge: After I started running off of the master boot branch my uber problem disappeared.

I then started seeing the “too many files open” bug as well, your patch fixed it for me, thanks!

@alandipert @estsauver I'm out now but will try to tidy up a PR later today

Nvm fixed it -_-

@mitchelkuijpers: what was the fix?

I have added the src/clj folder to the resources 😛

that fixes it, stupid trapperkeeper does not support aot

(Just trying to minimize the https://xkcd.com/979/ effect )

@estsauver: Haha you're right

@mitchelkuijpers: curious, why are you AOT-ing?

I am not anymore, because trapperkeeper does not support it

but the main reason is that the application starts a bit faster

ah, yes

i ask because i try to spread the good news about the web + war tasks, and detest AOT

Aha ok I don't exactly know what the web task is?

it allows you to generate servlets without AOT, by generating a web.xml file and putting it on the fileset that's compatible with https://github.com/tailrecursion/clojure-adapter-servlet

so if you are deploying to a servlet container you can both 1) not have to aot anything and 2) user uber --as-jars (much faster than default uber)

Oh thats nice!

i have been working on and off on something similar, except for cli

that provides a proxy -main that runs a build.boot in the jar, so we can distribute boot cli programs with no deps except jvm

@ragge: re: PR that sounds great, thanks!

Does that mean you could kind of use boot as an integral part of your distributed application?

well, you can already do that if you're brave enough

more that it would make it possible to distribute standalone jars easily

Hmm might need to look into that

I am currently building the distribution of this application http://dash.avisi.com/ with leiningen

Lol should fix that description of the page

But that is the reason I am looking at boot creating a tar.gz with custom stuff is very narly with leiningen

that looks really cool!

i can imagine producing debs and rpms also, with fpm in a jruby pod

yes definately

Had never seen fpm, thank you sir!

sure! :thumbsup:

I get some errors with instaparse

I think this is probably the bug: https://github.com/Engelberg/instaparse/issues/104

I guess the fix for now is making sure boot uses the same clojure version?

Yes that fixed it 😃

@alandipert: was actually considering doing something for deb as that's what we're mostly deploying... we're currently using fpm, but using it from embedded jruby in a pod sounds both a bit wicked, and very exciting. is that something someone is working on?

@alandipert: if not, it sounds like the perfect first proper task for me

@ragge: not that i know of, go for it!

@ragge: i made a ticket for it here https://github.com/boot-clj/boot/issues/224

it would be awesome if we could ship updates to boot.sh via package manager

@alandipert: just a question re the unpack-jar fn: what's the reason for catching this exception and just printing a warning? https://github.com/boot-clj/boot/blob/master/boot/pod/src/boot/pod.clj#L371

if exception is swallowed, that means a file that was in the jar to be unpacked will not be in the uberjar, which is pretty serious...

i would expect that to fail a task

that was due to issues like this https://github.com/boot-clj/boot/commit/abb64d20b760d81b19b1c32026af6173cab9377d

@ragge: ^^

hmm, ok

a comment there is probably appropriate

couldn't that be solved using exclude?

if you have a jar file with that issue

yes, i think so

yeah you're right

fix totally makes sense for that

but it felt weird in my case

just seeing a bunch of errors printed

but task succeeding

i mean the only thing you can do is exclude them, right?

so here we just do that for you and move on

probably does what most users mean

but warns so you can do something special if you need to

not sure I agree... as I said, it totally make sense in this case, but defaulting to automatically excluding files from your uberjar when an exception occurred somewhere in that fn...

hm

by this case I mean the case in the issue

other cases might be mine (too many open files), but could be other stuff too

just seemed like a dangerous default... was mostly asking to see if i should remove it in PR, but if you feel that should be the behaviour, I won't change it

hm

osx filesystem is case insensitive too, right?

they can be

not by default I eblieve

believe

err

they are insensitive by default yes

no, sensitve

haha

it would be a shame if we make it so that the user always needs to jump through hoops of excluding things to get uber to work

like if it's more likely to get exceptions than not i would say just warn

but if most cases can work without failing then throwing would be okat

s/okat/okay/

like the only fix for something that's not working in the uber task is excluding

well, it's only an issue when files do clash like in the issue... and currently boot makes a global decision that it's first-one-wins... how do you know that's what the user wants?

we don't, but the user only needs to intervene if it's not what they want

like in the issue, you'd get an empty directory, not the license file, am I right?

and in many, maybe even most cases they won't probably care

no, they'd get the file i think

because the fileset doesn't know about directories

it's like git

that's just windows refusing to create a file with the same name as the non-empty directory

ok

but even in the too-many-files case

there isn't any way to fix that other than excluding

of course you tell the user that their uber is not going to contain all the files

but the uesr can't fix it if excluding something doesn't fix it, right?

the too-many-files thing is a bug in boot

well, in this case they could bump filehandles, if that was the case, or fix boot

perhaps we differentiate the file exists exception from ioerror or whatever

file already exists should i think just warn

too me it just feels weird to not fail the task when an exception occurs like this (and if it's the license issue, let the user handle it, maybe print "if you want to exclude this failing file, add this /LICENSE to :exclude" etc...)

it's like the javac task would succeed even if compilation failed, but just print warnings

well userjars are hairy

i can imagine getting 50 or 60 file exists warnings

am i going to add 50 options to the thing then?

that would be really annoying

i don't have any data on that so I don't know

50 is a lot, that would be painful yes

uberjar is a packaging thing

which makes it a little different than a build task i think

like you do it once, and you pay attention, right?

not really

i guess that also means that you could take care to have proper exclusions too

i build on ci on every commit

if it started printing warnings but stil suceeding i wouldn't know

until I have an uberjar that doesn't work

well your ci could run the uberjar lol

but i see your point

to me, it's a task that fails to do what I wanted

sorry to bang on aboutu this

I'll just leave it as is for now

I have probably made my opinion known

haha no worries

i think it's fine to throw really

just exploring the implications

would you consider this a breaking change, or a bugfix?

changing to a throw?

i think it would also be a good idea to get some data on collisions, if possible

i agree the manual exclude solution is less ideal if it's common to have 50 conflicts

an additional flag to turn exceptions on is another option to consider

the uber task has been problematic from the start so i think it's maybe alright to change things around, because i think a lot of people are annoyed by how it currentlyworks

not sure flag is the right way to go

what other things are annoying people?

there are just more tickets about that task than anything else in boot i think

oh ok, might have a look around then to get a better understanding

we're converging slowly

@podviaznikov: there are still some references to meta.edn here: https://github.com/hashobject/perun#how-does-it-work

thanks, will remove them now

Btw. @podviaznikov I wrote a Boot task for LiveReload (https://github.com/Deraen/boot-livereload) it enables quite nice workflow for Perun: https://github.com/Deraen/deraen.github.io/blob/blog/build.boot#L34-L40

@juhoteperi: it’s pretty cool. I’ll try it

@podviaznikov: in some tasks it would be neat if the exceptions give some more information about what file caused it

It's not yet pushed to clojars as I'm waiting for clj-livereload PR to be merged.

@podviaznikov: also is there documentation on specifying the initial metadata in markdown files? Is that just something that comes with markdown-clj?

@martinklepsch: what exceptions do you see?

@martinklepsch: metadata is just “key:value” pairs. You can see example here: https://raw.githubusercontent.com/hashobject/blog.hashobject.com/master/resources/posts/20130603-clojure-webapp-with-https-support-on-amazon-beanstalk.md

and this function https://github.com/hashobject/perun/blob/master/src/io/perun/markdown.clj#L32 parses metadata

@podviaznikov: yeah, I figured that out, but I was used — as delimiters (jekyll) and surprised about the HTML comments

I think I can add support for jekyll style. I did that parses for my initial implementation 2 years ago and I don’t remember why I did it this way back then

created a task: https://github.com/hashobject/perun/issues/8.

I’m fine with either, was just confused initially, probably something that would be useful in the Readme/task docstring

@podviaznikov: re exceptions: getting them when having html inside markdown blockquotes, worked in kramdown but doesn’t with markdown-clj

I’ll try to reproduce error and see what can be improved for exception handling

also do you know other good markdown libraries in clojure?

No idea about markdown libs, I guess most just wrap the most popular Java lib (whichever that is)

As far as I know, Dmitri’s is the standard: https://github.com/yogthos/markdown-clj

However, in my latest project, I actually call out to the shell for multimarkdown.

podviaznikov: check this out: https://github.com/tailrecursion/boot-hoplon/blob/master/src/tailrecursion/hoplon/markdown.clj

it emits an AST basically, which you can extend

zero "rendering"

thanks @micha and @akiva

whoops: https://github.com/tailrecursion/boot-hoplon/blob/master/src/tailrecursion/boot_hoplon/markdown.clj

probably a more useful namespace

emits markdown as s-expressions

you can implement the operations yourself

my objective there was to use markdown as a general purpose s-expression markup language

so your "markdown" is compiled to a program

like i want to be able to reimplement something like <<foo bar>> for instance to be something different than what markdown usually does (the french style of quotation)

and in hoplon markdown is a program, so the AST is really what i want anyway

basically the functions in that namespace are what you can call from your own macros

they generate the lisp expressions corresponding to the markdown