Fork me on GitHub
#clojurescript
<
2015-11-15
>
dnolen00:11:44

Yeah just doesn't exist in 178

dnolen00:11:20

It could only come from older version

mfikes01:11:48

After a few years working on a compiler heavily based on immutability, David says, “I've got some time to kill. I know—I'll throw in a couple of fns to parallelize the whole shebang.” :)

johanatan03:11:59

Nobody knows the answer to my q?

johanatan03:11:01

@dnolen: what happens if eg there are 8 cores & 9 or more deps earlier than a common ns in the queue? Deadlock? Also, even if full deadlock doesn't occur there could be wasted opportunity for the blocked agents who could be working on ns further into the queue whose deps are already resolved.

dnolen04:11:55

@johanatan: can’t happen, already topologically sorted

dnolen04:11:53

as far as wasted opportunity yes this could be true, would need something that actually partitions distinct significant subtrees of work.

dnolen04:11:39

but my hunch is that it will probably be wash and improvements beyond this will be much more incremental.

dnolen04:11:20

in the end 30%-50% speed bump across a large spectrum of contemporary hardware for 20-30 lines of code is a clear victory simple_smile

johanatan04:11:15

@dnolen: ahh, cool. Ya, totally

johanatan04:11:12

@dnolen: there may be a simple solution to the wasted opportunity tho: like a priority queue of sub trees

johanatan04:11:43

Or just a skip list over the various largish chunks of tree

dnolen04:11:54

@johanatan: sure but I’m not gonna to work on this any more though simple_smile It’s now clear where this logic can go and we’ll happily take enhancement patches.

dnolen04:11:19

(well not work on it beyond error reporting issues for the current implementation anyway)

johanatan08:11:35

to answer my question re: deep prop gets/sets: aget and aset do accept multiple levels. Unfortunately I could find nothing better than (aset "the" "props" (clj->js (merge (js->clj (aget "the" "props")) {:new :props}))) for doing the merge / property update/addition.

dm309:11:24

there's https://github.com/purnam/purnam if you need a lot of interop

johanatan09:11:23

interop? Lol

johanatan09:11:35

Cool, I'll consider that if these things grow too much

dnolen11:11:57

@johanatan: aget and aset really isn’t for dealing with Objects, see goog.object

jonas14:11:53

Can we assume that “public" functions in cljs.core which are not in clojure.core are not implementation details? I’m thinking particularly of random-uuid. This function is cljs only and I’m not sure if it’s to be considered part of the public API?

dnolen15:11:42

@jonas: random-uuid is public

moxaj15:11:39

Why do I get "goog.require could not find: cljs.core$macros" with :whitespace optimization, and not with :none? Also, I don't think I'm even using that namespace ...

jonas15:11:20

@dnolen I know it’s public, that’s why I’m asking. I have previously considered all functions that exists both in clojure and cljs as set in stone (thanks to http://clojure.github.io/clojure/). Since cljs doesn’t have a separate api documentation it’s difficult to know what’s part of the public api and won’t change/be removed in future versions.

dnolen15:11:20

@jonas: I don’t remember the last time we removed something public from ClojureScript

dnolen15:11:24

cljs.core/format

dnolen15:11:28

is about it

dnolen15:11:35

we don’t remove things randomly and we never will

jonas15:11:03

That’s nice to hear, thanks!

jonas15:11:37

There’s an obvious advantage in having the two (clj and cljs) apis as much as possible the same.. so perhaps we can push for having random-uuid in clojure.core as well?

dnolen15:11:25

@jonas: sure I would talk to @alexmiller about that

dnolen15:11:08

@mfikes just tweaked the parallel build stuff, feel free to test simple_smile

mfikes15:11:23

@dnolen: Awesome… will do!

dnolen15:11:00

added proper error handling

dnolen15:11:13

this stuff should be good to go for the next release simple_smile

mfikes15:11:24

(Damn. Too much awesomeness surrounding ClojureScript today to keep up with.)

dnolen15:11:59

@mfikes: thanks for putting together the independent ns example

dnolen15:11:54

my suspicion is that the parallel stuff really takes off for bigger projects that have independent nses like that

mfikes15:11:51

@dnolen: Yes, my activity monitor shows 100% vs 2000%, so I think that there is about 20x the work being done… something fishy I’d like to understand...

dnolen15:11:47

there might also be some small heuristic to implement here that give more consistent behavior across a wide variety of project structures

dnolen15:11:57

something to think about

dnolen15:11:26

still we have a nice clean way to introduce such enhancements with little affect on anything else

mfikes15:11:33

@dnolen: I tested with your “error handling” patch. My iOS project didn’t break when recompiled with it, and the fith-postulate test runs properly (at the same speed as previously). simple_smile My CPU is actually running around 2200% and 2300%—its using all the cores so I want a 24x speedup. Gonna have to dig into that mystery.

dnolen15:11:22

Might just be I/O bound?

thheller16:11:40

@mfikes could you do me a favor and run lein cljsbuild once parallel a second time (without clean)?

thheller16:11:56

my machine isn't capable of handling your crazy setup

thheller16:11:26

well .. it is still running after about 30min 😛

mfikes16:11:18

@thheller: A second time without clean appears to work properly (it finishes quickly)

thheller16:11:47

@mfikes I'm interested in the time

thheller16:11:08

didn't expect any breakage

mfikes16:11:18

$ time lein cljsbuild once parallel
Compiling ClojureScript...

real	0m8.770s
user	0m39.605s
sys	0m2.806s

dnolen16:11:57

for non-cold builds I’ve seen that we lose a lot of time on reading EDN

dnolen16:11:14

will probably investigate switch to transit for analysis caches

dnolen16:11:33

I expect we can cut non-cold build time w/ zero changes by >10X

thheller16:11:16

@dnolen I'm aware of a couple of bugs related to caching ... but not excatly sure how/where to report them

thheller16:11:27

since the scenarios to trigger them are rather complex

thheller16:11:58

but everytime someone is told "did you try lein clean?"

thheller16:11:06

that is a manifestation of such a bug

dnolen16:11:14

yeah we need minimal cases

thheller16:11:15

to me that is unacceptable

thheller16:11:41

well the minimal case involves at least leiningen

dnolen16:11:47

there’s also a whole class of low hanging fruit around aborted builds

thheller16:11:49

and you don't like that

dnolen16:11:01

not going to look at anything involving lein for this problem

thheller16:11:07

lein is not the issue

thheller16:11:10

dependencies are

dnolen16:11:32

then you should be able to repro trivially without lein

thheller16:11:19

well yeah but the step by step gets much more confusing

dnolen16:11:56

I haven’t personally seen issues with caching in a very long time

dnolen16:11:11

the only cases I’m aware of, user changes dependency version or aborted build

thheller16:11:43

yeah changing a dependency is the trigger

thheller16:11:00

but that should still not require a lein clean ever ...

mfikes16:11:12

With debug-prn I can see that it plows through blocks of 20 or so namespaces at a time owing to them all being the same size. This gist shows that behavior, with me manually adding newlines between each chunk or wave: https://gist.github.com/mfikes/32efc0c717b141b6ea43

dnolen16:11:09

@thheller: then the steps to repro aren’t complicated are they? simple_smile

dnolen16:11:27

we have logic for recompiling JARs that change but obviously something is wrong

thheller16:11:59

yeah the issue is that it currently tries to determine whether a recompile is needed based off of one timestamp

thheller16:11:13

that isn't accurate enough

thheller16:11:41

need to account for the timestamp of all deps (eg. jars)

thheller16:11:35

eg. you compile today with om alpha20, then update to alpha22 and compile again

thheller16:11:48

since you compiled today .. last modified is today

thheller16:11:09

but alpha22 was released yesterday

thheller16:11:19

so no recompile is triggered

bherrmann16:11:50

(reposting from the clojure channel, since there are more clojurescript peeps here) Hi. I'd like to deploy a reagent project to tomcat. I created a "lein new reagent myapp" project, then I added the lein-ring plugin. Then I "uberwar" and deploy. But I get the "ClojureScript has not been compiled!" message.... despite it being compiled during the "lein ring uberwar" What am I missing?

thheller16:11:51

(no idea what actual om release dates are, just an example)

dnolen16:11:21

@thheller: sounds like a bug with cljs.closure/source-on-disk

dnolen16:11:36

not anything in cljs.analyzer or cljs.compiler

thheller16:11:41

I have no idea where the bugs are just how to trigger them (since I fixed it in shadow-build) which was using the same logic requires-compilation? or requires-analysis?

thheller16:11:43

a single timestamp is the issue

dnolen16:11:47

yeah those functions don’t matter

dnolen16:11:03

anything in JAR goes through source-on-disk

dnolen16:11:25

and that logic won’t copy out fresh stuff to disk

thheller16:11:41

you misunderstand

thheller16:11:04

well ... maybe .. dunno what sources-on-disk does

thheller16:11:14

but if it preserves the date of the jar

dnolen16:11:35

@thheller: you misunderstand that I understand what you are saying

dnolen16:11:41

and that bug is present in source-on-disk

dnolen16:11:46

which is what copies stuff out of JARs

thheller16:11:25

but why is copying files out of a JAR important?

thheller16:11:59

ah it copies with a new date

dnolen16:11:03

because if we fix source-on-disk the bug just goes away

thheller16:11:24

hmm no that seems to already compare the date

thheller16:11:00

ah ok got it

thheller16:11:20

well, that would fix a part of the issue since it doesn't account for macro changes

thheller16:11:32

but probably unlikely that a macro changes without the cljs changing

thheller16:11:46

excellent, I could not think of a simple description of the issue

moxaj20:11:45

Hey, anyone knows why I get "goog.require could not find cljs.core$macros" if I have [cljs.js :refer [eval js-eval empty-state]] in my ns require? Works with :none optimizations, but fails with :whitespace (compiles without warnings, but I get the error message when loading the document).

akjetma22:11:23

hey, anybody know if it’s possible to get figwheel to refresh when i change a non-cljs/css file? i am editing a plain js file and it would be awesome to have my reload-fn triggered on changes

jmmk22:11:26

How do I get a CLJS repl running in cursive? I don't need any reloading into a browser, I just want to be able to evaluate code for library development. Maybe this is a question for #C0744GXCJ... sorry just noticed that channel

bherrmann22:11:51

Jmmk, I think there is a REPL execution configuation

bherrmann22:11:05

you just create one, then run it

akjetma23:11:00

thank you @danielcompton. figured it out, am thoroughly enjoying it