Fork me on GitHub
#clojure-dev
<
2019-09-03
>
andy.fingerhut23:09:37

Still no definitive conclusions on the performance oddity I mentioned earlier, but I am currently inclined to think that the nature of the code being run there (nothing but iterating a loop count variable and comparing to see if it is done) leads to much higher variance in the performance of the JIT'ed code, vs. if something significant is being done in the body of the loop.

andy.fingerhut23:09:34

I did happen across this article today, by accident, where someone has done a huge amount of careful work running a bunch of benchmarks and using statistical methods to look for cases where performance with a JIT is not of the typically expected "slow for the first N warm-up runs, then consistently fast for all later runs" form: https://tratt.net/laurie/blog/entries/why_arent_more_users_more_happy_with_our_vms_part_1.html

andy.fingerhut23:09:53

although before you go off and spend whatever time reading that, note that a lot of the performance variations they measured were in single digit percentages. But typically for much longer-running benchmark programs on the order of many milliseconds to seconds.