Fork me on GitHub
#cljs-dev
<
2015-12-09
>
mfikes16:12:18

There is an interesting issue related to var resolution for :referd symbols with a patch that I’d be interested in comments in http://dev.clojure.org/jira/browse/CLJS-1506

dnolen16:12:25

@mfikes: looks reasonable to me

mfikes16:12:58

@dnolen: Yeah, it looks fairy cut-n-dry, and makes it match the similar stuff in :else. Scary to me simply because of where the change is.

richiardiandrea16:12:33

oh, that's a problem I had too 😄

thheller20:12:32

curious why you do the +2 since mfikes experiments showed that there are very little gains beyond 4 threads?

dnolen20:12:59

@thheller: that’s not true

dnolen20:12:13

you just stop seeing linear after that point

dnolen20:12:38

plus that was only @mfikes feedback, will still want to hear from more people with actual big projects

thheller20:12:39

yeah right but assuming that all cores are busy

thheller20:12:05

there wil be 2 threads waiting

dnolen20:12:05

and also feedback about modern multicore hardware

mfikes20:12:06

My hunch is Gene Amdahl was on to something. simple_smile If 15% truly can’t be parallelized, you end up asymptotically approaching 6.5× no matter how many cores you have. Of course, the real picture is probably much more nuanced with respect to memory bandwidth and other factors. My guess is that it is hard to use lots of cores on any system that has even a small bit that shares any access to something.

thheller20:12:28

@mfikes I'm curious why compiling with 1 thread takes x sec per file

thheller20:12:43

but multiple threads take (* 3 x) per file

dnolen20:12:20

@mfikes: I withhold judgement until we get some results from on 12 core Mac Pro or a AWS Compute Cluster 😛

mfikes20:12:44

@dnolen: My results are fro a 12-core Mac Pro

mfikes20:12:59

Albeit one from 2012

dnolen20:12:02

@mfikes: I thought you said it was one of the hexa-core’s not a Xeon Garbage Can thing

mfikes20:12:13

Yeah, an old dual-hexacore

mfikes20:12:34

Garbage can would be nice to see the results for simple_smile

thheller20:12:09

the can is basically from 2012

thheller20:12:21

didn't see a CPU update since release

thheller20:12:36

I'm waiting for an update 😉

thheller20:12:28

ok nvm Dec 2013 guess that counts as a full generation or two

mfikes20:12:05

For the life of me, I never found out what the contention is. I’m assuming it is memory bandwidth or something hard to see in a profiler.

thheller20:12:34

@mfikes you saw the same slowdown for individual file compile times on your machine correct?

thheller20:12:42

eg. no parallel takes 2sec per file

thheller20:12:47

parallel takes 5+ per file?

thheller20:12:49

it might be a CPU cache related thing

mfikes20:12:22

@thheller: If I understand you, yes.

mfikes20:12:04

I put my repo online so anyone can repro

thheller20:12:23

yeah, I played with you repo but it breaks my machine

thheller20:12:26

not powerful enough 😉

mfikes21:12:34

I suppose you need at least 6 cores to get a meaningful deviation away from linearity. Perhaps the cool thing is that the majority of dev machines out there will be in the linear regime. simple_smile

thheller21:12:47

dunno the overall gain is minimal unless you are always lein clean before building

thheller21:12:06

if you are building incrementally there are usually only a couple of files to be compiled

thheller21:12:14

in which case threads actually hurt sometimes

thheller21:12:34

but once the JVM is up and warmed up threads still win

dnolen21:12:52

@thheller: this is definitely true, but it’s still a good gain for cold builds when they happen and they do happen and prod builds, and far as I’ve seen the hit for incremental builds is quite minimal

thheller21:12:54

thread definitely win on cold builds

thheller21:12:11

my prod builds are incremental 😉

thheller21:12:26

why I wrote the ReplaceCLJSConstants closure compiler pass 😛

dnolen21:12:32

sure but not everyone uses shadow-build

dnolen21:12:39

and many people never will

thheller21:12:13

well, we just need to get that stuff into cljs 😉

thheller21:12:39

nothing shadow-build related in it really

dnolen21:12:22

given the track record, it’s usually just a matter of time simple_smile

dnolen21:12:44

reliable incremental is certainly high on the list of priorities