Fork me on GitHub
#cljs-dev
<
2018-08-31
>
mfikes02:08:11

I’ve reproduced that result locally perhaps half a dozen times now, and it is also appearing in the Coal Mine Travis CI results with the first part of Coal Mine previously taking around 14 minutes and now taking around 10 minutes. Perhaps something about the optimization plays well with HotSpot after the compiler has warmed up (theory based on the Coal Mine compilations being rather lengthy).

mfikes02:08:15

It could also be something about that first set of 59 problems that benefits from the optimizations.

mfikes10:08:07

If tickets are looked at today, these two are pretty cut-n-dry one-liners that would get unit tests / Canary passing again: https://dev.clojure.org/jira/browse/CLJS-2872 https://dev.clojure.org/jira/browse/CLJS-2877

dnolen18:08:38

@mfikes approved, go for it

mfikes19:08:55

I’ve started a project to benchmark compilation over time, using the same hardware and source bing compiled, but with the compiler varying. It will take some time to fill in the single-threaded numbers. Also, if you want to bench a candidate change, the benchmarks commands use git deps and are thus standalone and reproducible. https://github.com/mfikes/cljs-perf

dnolen19:08:23

gonna do some more profiling today

mfikes19:08:46

Cool. Those “micro” opts turned out to be significant 🙂

dnolen19:08:15

I think there’s a lot unnecessary overhead when generating the output file

dnolen19:08:26

Clojure’s printing machinery is gratuitous when all we’re doing is generating JS source

mfikes19:08:25

Ahh, right, we are mostly appending strings, and want to do that as fast as possible

dnolen19:08:38

we don’t need Clojure printing at that point

mfikes19:08:09

Wow, maybe a Java StringBuffer per output file. It would be gross, but probably fast

dnolen19:08:28

that might help but just removing the obvious problems first

dnolen19:08:02

then will reprofile with YourKit

dnolen19:08:20

pushed another micropt

dnolen19:08:22

reprofiling now

mfikes19:08:09

Yeah, focusing on the O part of I/O makes some sense: Watching it compile, it writes 1 to 2 orders of magnitude more bytes than the compiler reads.

dnolen20:08:47

heh, there’s a lot of microoptimizations do it seems …

dnolen20:08:15

another significant small one was gratuitous usage of binding

dnolen20:08:00

my fault since I think I did this to make testing “easier”

dnolen20:08:17

pushed another micro-opt + dropped gratuitous binding in analyzer

mfikes22:08:50

Wow, that last commit did extremely well in single-threaded mode. https://github.com/mfikes/cljs-perf

dnolen23:08:41

pushed two more micro-optimizations - been a long time since I’ve seen the :none compile portion of script/test < 10s, for that it looks like close to 10% gain

dnolen23:08:58

whoa I didn’t realize coal mine was so huge 🙂

dnolen23:08:11

hrm interesting, it doesn’t look like for the end user at least that compiler perf has regressed that badly

dnolen23:08:18

I just checked 1.7.10 (to be fair pre-parallel build), which is 3+ years old, master is faster for compiling the basic sources

mfikes23:08:41

Hah single threaded compilation is currently 1.66 times faster than 1.10.339 (without that latest commit, which I’m going to test now.) 🙂

mfikes23:08:14

Yeah, Coal Mine is more than a quarter million lines of code, and I’m not done yet. It feels like I’m working on the OED.