Fork me on GitHub
#cljs-dev
<
2018-11-20
>
richiardiandrea16:11:50

Today I had a dream - or nightmare it depends 😃 - what is the Cljs compiler became native up until shelling out to the JVM for the optimization phase? I don't know too much about the internals there but is this gaining something in terms of speed? I know that this would mean the REPL would be a completely separate thing now

thheller16:11:48

native what? 😛

borkdude16:11:10

rebuild it in Rust

borkdude16:11:24

or maybe use Joker, it already kind of works 😉

thheller16:11:25

so basically restart from scratch to gain a second or so. very doubtful that'd be useful 😉

richiardiandrea16:11:12

Yeah of course I am not really aware of the seconds to be gained

richiardiandrea16:11:34

Because the GCC step might be now intertwined with other things

richiardiandrea16:11:57

Just wanted to say that I really had a dream with that lol kinda funny

richiardiandrea17:11:44

@thheller is startup time only 1 second for you? Genuinely asking, never profiled it but mine feels more like 5 before I see "Comping..." in Shadow

thheller17:11:10

no idea. last time I measured it was somewhere closer to 3sec (AOT version). don't care much since server-mode makes it zero 😉

richiardiandrea17:11:51

well except in CI

richiardiandrea17:11:29

we spoke about this actually, I should probably have a Shadow as a service

richiardiandrea17:11:45

now I am launching instance one per folder

thheller17:11:48

do you care about 3sec on CI?

richiardiandrea17:11:02

our CI at the moment completes in 20 minutes and Cljs is losing big time compared to TypeScript 5-7 mins

richiardiandrea17:11:25

not a big deal for me, but I need to protect my investment from detractors 😄

thheller17:11:39

persist the cache between builds and CLJS will easily beat TS

thheller17:11:05

(well depends on the time closure takes)

richiardiandrea17:11:06

yep I know, except that this means to build a Docker image in Azure where we are

richiardiandrea17:11:30

and folks here don't have to do that, I don't have time for doing that, so it would be programming in anger 😄

richiardiandrea17:11:49

in any case it will happen sooner or later

juhoteperi17:11:32

That sounds quite slow. "Big and slow" projects take like 6 minutes for me, 3min 50sec for unit/integration tests, 1min 30sec for Cljs advanced build and uberjar build, 35sec push to dockerhub (could be optimized to few seconds)

thheller17:11:09

has anyone here been able to compile the Closure Compiler in IntelliJ? I just can't get it compiling properly

richiardiandrea17:11:28

our builds only range from 40-80 seconds (multiply that by ten projects) - the most of the time is spent downloading dependencies

richiardiandrea17:11:24

what I am saying is that we are already a niche language - everything that you need to add on top of things is seen as "why do I need that, TypeScript doesn't need that"

borkdude17:11:33

downloading dependencies: cache - probably stating the obvious. not on CircleCI or anything like that?

😁 4
thheller17:11:52

sounds to me like your TS projects are cheating and caching node_modules 😉

richiardiandrea17:11:06

@thheller well that might actually be 😄 @borkdude I wish I were on Circle

richiardiandrea18:11:17

I opened that issue on shadow but I see no further discussion on how to solve is needed 😄

lilactown18:11:23

the three biggest things we did for CI was cache dependencies (node_modules and m2), cache the .shadow-cljs folder, and make each build run sequentially

thheller20:11:18

@roman01la it emits v3 yes. not sure what fully compatible means though. there isn't much to it really.

Roman Liutikov21:11:17

Ok, thanks. I got invalid prefix error from Mozilla’s parser but couldn’t reproduce it again

darwin21:11:46

well, “emitting v3” does necessary mean “always produces correct output” and Mozilla’s parser could have bugs, hard to tell

thheller22:11:10

@roman01la the source maps in optimized builds are incomplete since they miss any cljsjs maps. besides that some parts are not properly mapped, especially macro related code so there will be "gaps"

mfikes22:11:08

@thheller Tangentially related—do you have an implementation of https://dev.clojure.org/jira/browse/CLJS-2895 in shadow-cljs?

thheller22:11:55

no. I instead just convert the data once and only store the VLQ encoded version

thheller22:11:05

(this significantly increased cache performance. 2.5s -> 250ms for cljs.core alone)

mfikes22:11:19

Wow. (And that is essentially storing the “unculled”, fairly verbose, info.)

thheller22:11:28

the 2.5s version was storing the raw source map data directly yes, now its just the vlq encoded mappings which is basically the culled version

mfikes22:11:27

Oh. Did you find certain things that could be culled? (Or at least de-duplicated?)

thheller22:11:45

the conversion process already seems to cull most of it anyways

thheller22:11:22

I looked at a bunch of maps via https://sokra.github.io/source-map-visualization/ and it all seemed ok with some minor gaps

mfikes22:11:00

Cool. Sounds like something along the lines of CLJS-2895 can help general perf. (Source mapping is not horribly expensive, but still feels like an area worth optimizing in some way.)

thheller22:11:53

yeah definitely. it looks like a lot of info is thrown away in the convrsion process anyways so not collecting it in the first place may improve performance

mfikes22:11:06

Re-reading that ticket-it looks like a 1.1 speedup is possible

thheller22:11:48

as long as the gaps don't get larger that sounds good 🙂