Clojurians Log v2
tools-build
The tools build uber task can be a little slow - looking at the code, it’s highly sequential, handling conflicts one by one. However, I wonder if there might be some way to pre-process and find conflicts ahead of time, allowing non-conflicting files to be processed in parallel? Is this even a good idea?
the most common slowdown i've seen is having a security app on your machine that scans files as they're written. for me, building with that app enabled takes minimum 30s. without it, like 5s
sheer shock and awe: you have 110k files in your jar?
we do! it’s a big project
think it’s not that unusual for large clj projects?
I’ve trimmed almost all the fat I can from this too
i guess i'll check my old prod app. should build that every once and awhile anyways
but okay to start: maybe just copy the entire uberjar namespace into your project and lets tweak from there
then once you are done tweaking and profiling see if there is anything worth merging back up
I stared a fork already - but it had some odd bugs, and was pulling it in as a git dep https://github.com/dazld/tools.build/tree/parallel-uber
instead of forking the library itself i'd just copy that one file and change the namespace to your.company.tools.build.tasks.uber
this is also pretty easy
but yep, could do that too
since i'd say 50/50 the conclusion is going to be "yeah you get your own special snowflake"
tbh - even when the parallel build was up, even if buggy, it wasn’t that much faster
but I do think there’s opportunity here
fortunately, that’s not an issue for us on CI or locally (afaik) - we have some 110k files in our JAR, so it’s not surprising it takes some time to build I suppose.
> jar tf uber.jar | wc -l
109901it all adds up!