I cannot believe it ...

I got this to compile https://github.com/swannodette/es6-demo/blob/closure-master/src/libs/NodeStuff.js

unconfirmed whether it works, but it actually compiled

and all the dependencies that were used were correctly found

https://github.com/swannodette/es6-demo/blob/closure-master/script/build.clj

a little bit hackish at the moment - but very promising

oh wow...

React too?

yeah

React from node_modules

but this confirms my suspicion that trying to integrate with CommonJS NPM modules is kind of dead end

most exports are dynamic

so compiling 3rd party CommonJS is a crapshoot

ES6 just seems way more promising here

static import & export

however there’s still some value for people who want to use NPM as their distribution mechanism

instead of JARs where the JS files they write are Closure CommonJS friendly

and Node.js friendly at the same time

hrm or maybe not … I don’t know yet - there maybe something about how we evaluate goog.require under Node.js that is wrong.

tested latest master (367e2f) against a 20K line reagent/re-frame app and all works fine slightly_smiling_face Noticed initial build time on :simple optimisations increased avg 34 seconds (80->114s) and rebuilding on changes increased avg 10 seconds (~20->30s). @dnolen

@superstructor I don’t know about :simple since that’s Closure time

but rebuilding on changes is interesting, will need to hear more information

should I try optimisations :none and time it again ?

is time reported by the ClojureScript compiler or something else?

if something else it’s really not super useful

since it’s not something I’ll really consider looking at

it needs to be ClojureScript time only

WOW

ok the node_modules thing works - just a problem with how we bootstrap Node.js

so that needs some thought but it seeems surmountable

@dnolen time reported by :compiler-stats true and I think the final “elapsed time” is part of our boot-clj tasks; e.g.

Compiling ClojureScript...
• js/app.js
WARNING: :preloads should only be specified with :none optimizations
Compile sources, elapsed time: 545.27221 msecs
Compile sources, elapsed time: 153.117463 msecs
Optimizing with Google Closure Compiler, elapsed time: 11372.209861 msecs
Optimizing 490 sources, elapsed time: 28017.807772 msecs
Writing target dir(s)...
Elapsed time: 30.921 sec

@superstructor like I said anything in Google Closure time is not something we can do too much about

it’s also nearly impossible to swap out Google Closure since they make breaking changes and we are often pinned to a particular one since we integrate so tightly

@superstructor I’d be more interested in :none time

@dnolen ok thanks I’ll do some tests on :none time and see if there is a substantial difference between versions.

A scary commit 🙂 https://github.com/clojure/clojurescript/commit/59a7f265fac02c931cc3d5615727da861db62e3b

if you use Node.js you should definitely give it a spin

prep stuff for node_modules support

welp

I just advanced compiled a project that uses node_modules

and React works

@dnolen so on :none the compile sources phase actually appears to be slightly faster (if anything) going from 293 -> master. Just the Google Closure optimisation phase on :simple and above that appears to have slowed down, so I understand thats not in scope of CLJS. Thanks :thumbsup:

@superstructor glad to hear it

wow

so this stuff really just works

but externs are definitely going to be an issue - but funny cljsjs kind of solves that problem

you can include those deps just for the externs

heh

using Facebooks own optimized dependencies

53K gzipped for React + React DOM Server

advanced compiling ClojureScript from node_modules, including a good chunk of cljs core via the usage of a persistent vector and the println stuff - 66K gzipped

exciting times ahead, exciting times

Almost fits on a Commodore 64!

ha 🙂

curious why you had to remove cljs.nodejs/process and cljs.nodejs/require

that may be a breaking change for people on Node

@anmonteiro yes it will be a breaking change

basically the problem is that Google Closure stuff has to globally required

when we globally require we need to disable goofy JS detection of the Node environment

we do this by erasing module and exports

oh right. I know what you're talking about

I do the same in Lumo

@dnolen btw, I just discovered why Lumo isn't building. Got one more patch before tomorrow's release. Attaching it now

sweet!

I don’t want to break what people are doing with Node.js too much, so people need to try it

just want to see what the balance is

if it turns out these changes are too aggressive we’ll make some kind of compromise

but so far I’m thinking that being able to consume Node.js deps without losing your mind is a huge win

http://dev.clojure.org/jira/browse/CLJS-1909

A TL;DR view of CLJS-1909 is that is strangely reaches into macro-land, and then accidentally back out into runtime.

@dnolen right. fortunately these breaking changes are fairly easy to spot

Right. As @mfikes said it's an edge case that only occurs in self-host

the other cool thing is that when the Nodes JS stuff resolution lands for ES6

António’s CLJS-1909 patch sidesteps the issue

there’s nothing for us to do

which is why it was hard to figure out

I’ve banging my head all week on how people could write ES6 modules

like import as React from “react”

but now the answer is obvious

it will just work

this is a game changer. I'm really excited to try it out

hopefully I'll have some time over the weekend

yes I think so

between this & externs inference our integration story improved by at least an order of magnitude

yes, you've been hard at work

Sounds like some well-received blog posts are in order on the subjects 🙂

haha yeah

Never enough time 🙂

it’s going to take a while to really understand what the best way forward is

but at least we know it’s A) possible B) likely practical

@dnolen btw, I think this patch still makes sense: http://dev.clojure.org/jira/browse/CLJS-1822

but I wouldn't recommend putting it in this release. Probably need to try it out again and report back

ok, thanks will take a look tomorrow

Funny because at our Vancouver meetup today we where using nodejs/require a lot: https://github.com/paullucas/lumo-misc/blob/master/http.cljs

I guess it won't be possible anymore

the use case in question is consuming e.g. React

@richiardiandrea that thing was annoying anyway

from the very few months of ClojureScript

everything about how we interact with Node.js kinda stinks

they have a dev bundle which doesn't strip away some checks when compiling with :advanced

You know, actually yes 👍

@richiardiandrea I’d like the Node.js integration to be a lot simpler

if we’re going to break a few minor things, it’s a good time to consider how to make it better

At work we are going to probably write a big chunk of code using cljs on Node.js, so I am very excited about this, thanks

well making Node.js target simpler is one thing

if you don’t need to target the browser require is fine

and I don’t want to break that - so people should test

js/require is what I mean

the other thing is that people use NPM for doing client side stuff

and we don’t have good integration with that

that’s the thing I’m actually working on

Oh okay, will test that, got it

if you’re doing everything in Node.js for server side - life is pretty simple already

a lot less to worry

We are about to, and actually estimating if it is a good choice, but because we are targeting AWS lamdba it is kind of the only choice if you want to keep using Clojure

right

well try master please, and report 🙂

So I have to say that these new patches are actually solidifying my proposal (which I am kind of pushing) and could not come in a better moment 😀

Will do

@richiardiandrea could you run Lumo scripts on AWS for example?

I'm not suggesting you do, just asking because I'm unfamiliar with it and don't know if you can run third party binaries

@anmonteiro on an EC2 instance yes, the code above was a little exercise during the meetup

On lamdba you run js

I meant AWS Lambda

I know you can run on EC2, sorry

I went to double check but no: > AWS Lambda supports code written in Node.js (JavaScript), Python, Java (Java 8 compatible), and C# (using the .NET Core runtime). Your code can include existing libraries, even native ones. Please read our documentation on using Node.js, Python, Java, and C#.

👍 thanks

http://dev.clojure.org/jira/browse/CLJS-1911 is worth looking at closely before release

So cool to read up on all the progress done here. Looking forward to the day where we can say CLJS React apps are smaller than JS React apps hehe

Wondering how hard it would be to run yarn inside nashorn...

@anmonteiro: AWS Lambda calls to one of python, js, java, but there's nothing stopping you from shelling out to Lumo

@martinklepsch technically they already are 🙂 based on my test yesterday React+ReactDOMServer is about 8K smaller by going through Google Closure

@anmonteiro yes I was are of require issue but I didn’t have a good answer for that - that patch looks fine to me

@mfikes ^

Cool

@martinklepsch stuff like Yarn unlikely to work because they haven’t thought about the “bootstrap” problem

I/O stuff etc., Babel ran in browser always so it’s easier to integrate

If Oracle had pursued Avatar (Node.js API for Nashorn) we’d be in a different spot

Ah I see. Somehow thought Nashorn would be exactly that, but it’s “just" JS

yes with actually very good bindings into Java

significantly fancier than Rhino

There seem to be some other things out there though… http://dynjs.org/ & http://nodyn.io/

@martinklepsch I looked at that before and I suspect it’s not good compared to Nashorn (wrt. to quality completeness of JS runtime)

might be, either way, probably not “the pragmatic approach” just to get rid of the node dependency for development

I mean maybe it takes 2 minutes to eval Babel.js in that thing 🙂

I agree worth looking at, but my experience with non-spec compliant JS runtimes has been pretty bad

@dnolen: awesome. We can also just bind require to global.require outside the function, but in Mikes's patch it is right next to the problem so might be easier to reason about why it's needed

@anmonteiro I already did that on master

I believe this approach should cover us

the issue is that we never want to use require for loading Google Closure stuff

but we don’t want code that uses require directly to break

Ah right. On phone, sorry. Yes it's fine lile that, just saw it

@dnolen: consider also applying CLJS-1910 to get the bootstrap tests running again

@anmonteiro shouldn’t that work? with require fix?

oh oops

It uses cljs.nodejs/require which you commented out

I see

yeah OK

👍

Not sure if the fix is to bring those back or to apply mikes's patch

yeah messing around with it now

I think we’ll just bring them back

ok now would be a good time to test master 🙂

going to work on an “advanced" part of the JavaScript modules http://clojurescript.org guide that shows how to consume node_module

Cool. I'll give it a spin in about an hour

hopefully it inspires people to think how make it easier - I don’t think it will be very hard

@dnolen I smoke-tested master and it looks OK to me. 👍

@mfikes excellent!

I’m deprecating the ClojureScript wiki, please direct all edits to the ClojureScript site, thanks https://github.com/clojure/clojurescript-site

also if you know how to use master, please do so on your projects

a massive set of changes is about to land

@dnolen https://github.com/clojure/clojurescript/blob/7d856d7f04dde5e450462cbf24866011803f47a6/src/main/clojure/cljs/closure.clj#L1921

oops heh

yeah will fix

bug there: remove opts, it's already threaded through

fixed master thanks

one other thing. one recent patch added timing info for every file that's compiled

in my computer it shows that compiling cljs.core$macros takes about 27 secs

is this to be expected?

it's by far where the compiler is wasting the greatest amount of time

apart from that, Lumo works great with latest master

builds successfully, every test passes, etc.

@anmonteiro I would expect that file to take some time

but I do wonder where that time is spent

I’ve long had a suspicion that there’s perf problem with reading .cljc files

27 seconds on a fairly good computer. takes about 40 on my 2.5-year old mac

but I haven’t had anytime to look into this

that would make sense

yeah om.next compilation also got a lot slower

when we switched to a single file

@dnolen that would also explain the thing I showed you at EuroClojure

exactly

so it’s something to look into

I suspect it’s simple

let's open a ticket so we don't forget

on it

thanks

http://dev.clojure.org/jira/browse/CLJS-1913

for every numerology fan out there

lol

lol you guys are funny 🙂

I will need to update replumb now with all these changes 😄

tested a "standard" boot project against master and no problems so far

(also used master for this project btw, no issues, but also very small project)

@martinklepsch it seems moderately useful though it should probably have a lot of caveats

more than anything it may show that some libraries have DCE problems

got some regressions in my tests, git bisect identified this commit: https://github.com/clojure/clojurescript/commit/101d7d9e03e90518e6769781dd33fbe6387d2d44 it is probably my fault, FYI, investigating it now

found the issue in my code (wasn't obvious): https://github.com/binaryage/cljs-devtools/commit/3267356be0d2d5d4084191f104cb80c239d2f6b6

I think it will be worth noting in release notes that sequable? changed and people might want to review its usage

also DCE stopped working in in my cljs-devtools tests -> going for another bisect hunt

according to git bisect this commit prevents DCE in cljs-devtools: https://github.com/clojure/clojurescript/commit/0560106fe8576fd7029d86e61be09cd1f246bfe0

this is the test project which gets compiled: https://github.com/binaryage/cljs-devtools/tree/master/test/src/dead-code/devtools this is the compiler config: https://github.com/binaryage/cljs-devtools/blob/master/project.clj#L73-L82 this is lein task to run it lein test-dead-code: https://github.com/binaryage/cljs-devtools/blob/master/project.clj#L107-L109

confirmed, validated it by hand, just to be sure

44d5e56b is last one which works

@darwin unless you’re inferring externs I don’t see how that’s possible

compiler output didn’t change in that commit

re: seqable? - noted

I’m not inferring externs there, it is really strange, don’t understand that commit (yet), just reporting what I’m seeing 🙂

sure, but I’m skeptical about the conclusion w/o more information

maybe it broke normal inferring?

https://github.com/binaryage/cljs-devtools/blob/master/test/src/dead-code/devtools/dead_code/core.cljs

my DCE tests rely on ^boolean inferring for :closure-defines

also doesn’t sound likely

so far, I was able to narrow it to this line: https://github.com/clojure/clojurescript/commit/0560106fe8576fd7029d86e61be09cd1f246bfe0#diff-97c50fd669986fb95db3957c1bab83baR769 added some debug printing, it definitely affects my existing code which has no externs inferring printing !!! sym -> pre, outputs stuff like this: https://gist.github.com/darwin/4832baf286287d94c32a32443fa8b60f

so it looks to me that newly code emitted due to (new) existence of :tag prevents DCE

(in my case)

I could try to print generated source before it gets passed to closure compiler and maybe compare it to old version if that would be helpful

@darwin I do not understand why your problem even matters

you need to explain why :tag is preventing DCE

if you’re doing something funny then it’s not something we care about anyway

I only care if DCE is affected for normal code

well, DCE worked in 1.9.293, I upgraded to master, it broke, the test project it dead simple, I don’t see obvious error on my side

this is not super important, some people rely on cljs-devtools to be elided by DCE in advanced builds, this will stop working for them

I personally don’t include clsj-devtools in advanced builds (via a separate cljsbuild profile configuration)

this is what I mean this is a non-problem

ok, fine with me, just came here to report regressions

right but I only care about DCE in general

not in your project

so if you can explain how this would affect DCE for everyone

then I’m interested

so far I do not understand how that could be possible

but maybe you know something I don’t 🙂

at this point I don’t understand compiler internals enough to really answer it, sorry 🙂

let’s see how the new release affects DCE for other people

we have a potential candidate commit/line for further investigation

ok sounds good

@dnolen just saw you doing some work with processes. I think you forgot about clojure.java.shell/sh which does what you are implementing basically

buffering streams, exit codes, etc

@thheller yeah I’m just not familiar with that ns, and it’s not really important now, I have something working

@darwin I just tried the trivial console.log hello world smoke test - DCE seems OK here

dnolen: ok, I also tested cljs-oops which has multiple projects testing DCE and all worked, but none of them is testing a code which should get elided by :closure-defines via false goog.DEBUG, which is probably the case which stopped working

oh hrm

the type of goog.DEBUG

just saying (sh "node" "args" "go" "here") => {:exit Int, :out String, :err String}

@darwin and that’s even though you’ve hinted boolean?

look here, that’s my test project: https://github.com/binaryage/cljs-devtools/tree/master/test/src/dead-code/devtools

ah, mean this one: https://github.com/binaryage/cljs-devtools/blob/master/test/src/dead-code/devtools/dead_code/core.cljs

@darwin js/goog.DEBUG is bad

I need more information

the one I care about is ^boolean goog.DEBUG

ok, I will test them one-by-one and get back to you

basically what happens is now is js is a kind of taint

so you don’t want to use unless really need it

we automatically handle goog

do I need to require it somehow?

you don’t need to require it

it should be a special case

ok

confirmed, only js/goog.DEBUG is causing DCE to stop working, goog/DEBUG is fine

@darwin great

@darwin thanks for looking into that and bearing with me 🙂

no problem 🙂 I’m glad we got it resolved

goog.reflect has some interesting things in it: https://github.com/google/closure-library/blob/master/closure/goog/reflect/reflect.js

(aget obj (goog.reflect/objectProperty "string_prop_name" obj)) allows prop-munging of property names, but can still access by string

goog.reflect/cache lets you use an object as a side-effect-free cache without preventing DCE

btw. this mandatory ^boolean thing reminds me this ticket, which fixes it (and hopefully more): http://dev.clojure.org/jira/browse/CLJS-1615

also the tests there would catch the regression 🙂

@favila whoa! 🙂

@darwin I tried that exact same thing >4 years ago

I think I saw a non-trivial bump in code size

ok, that’s a fair point, let’s hope for closure compiler to get smarter about our truth_ function

in future

unless I hear something ClojureScript 1.9.454 is about to be a thign

tested on a project, compiles fine with advanced optimizations output bundle got larger for ~0.5KB, changed from 797460 bytes to 797965 bytes

heh

I’m also getting compiler warnings Use of undeclared Var private.project/js which is really just this calls (js/Date.). There 7 calls, but I’m getting only 3 warnings, don’t know if it’s helpful

that doesn’t mean anything to me right now

need a lot more information

nevermind, it’s gone

@dnolen I know it is marked experimental but I'm really skeptical about the whole node_modules stuff. There are a bunch of sort of non-official things in the js build world that this completely ignores.

@thheller what’s there to be skeptical about

do you really intent to fully support all these things>

this something that other people already use directly w/ Google Closure Compiler

one really simple issue: react vs react.min

what you get off node_modules is the dev build

you need to run it through webpack or alternates

you can use Babel transforms

with process.env set

you don’t need webpack for that

all these things are boring details that people can sort out for themselves 🙂

the point is now they can

and ClojureScript doesn’t really care one way or the other

well that is my point

it SHOULD care

you care

but I do not care 🙂

well the user should care

ie. shipping dev react into production is not a good thing

they just want to use something - now they can

what I mean somebody else can figure out all that stuff

i.e. Webpack for ClojureScript

now you can build that because ClojureScript provides all the various bits

so basically your only goal is to let people call into babel or others via js-transforms?

people can do whatever they want

and no further?

I don’t know and I don’t want to know

it’s like REPLs

who cares how people use these things

as long as it helps them solve the problem they want to solve

well .. I also care a lot about the final result being optimal not just "solved"

I think you misunderstand what the goal here is

I’m not interesting in “optimal” anything

only enabling the community to sort it out for themselves

dealing with the wider JS ecosystem is not purely a technological one with a simple answer

it’s primarily a social one that’s going to require a lot of effort from many parties

the changes to ClojureScript are just about letting others do that work with less friction

ok I guess that is a worthy goal

just a bit skeptical about all this being in core vs a library

sometimes you have to do the skunkwork to show that it can be done & practical

if somebody gets inspired by this stuff and does something 10X better - we accomplished our goals

Figwheel is a pretty good example of that IMO

and I suspect what ClojureScript does arounds JS modules can easily be trumped

yes that is great but you also still have people using or whatever that ns is called

@dnolen hah, looks like the ClojureScript release missed this by a couple days: https://github.com/google/closure-compiler/pull/2130#issuecomment-275801425

so there is a risk of collecting too much dust

> This has been submitted internally & will appear in github on our daily export.

@thheller removing stuff is a completely different conversation

we’re not going to do that - zero value

@anmonteiro yep I’ve been following that one very cloesly

@anmonteiro but no idea when that will get into a release

and when it does nothing for us to do really

hopefully will be in the February release

@thheller far as I can tell the most successful things in technology are pretty darn dusty

HTML, JavaScript, CSS, Java

it’s not a compelling argument

yeah I guess

@anmonteiro fingers crossed, soon as that’s in we can fix the ugly ES6 example

having to avoid import for the deps is pretty ugly

I think it optimal case Closure should be able to optimize JS modules as well or better as Webpack

https://clojurescript.org/guides/javascript-modules has some invalid code

:verbose true])

@juhoteperi it’s already better

in the guide example you will build React with all the invariants stuff

and the output is like 8K smaller

Closure can already remove code where if condition is always false, etc, so it should be probably possible to define process.ENV as constant or something

so somebody should looking into eliding that stuff and measuring

@juhoteperi yes, in fact you could probably hack it with goog.define in CLJS

so many things to try 🙂

@thheller fixed thanks

@juhoteperi I don’t know if you saw earlier but you were right about the package.json stuff

fortunately Google Closure does the right thing internally

but we needed module-deps to compute what to pass to Closure

Okay

that’s something we could probably replace with some Clojure but I’m lazy

One thing that I hit when trying this a month or two ago, was that Closure "scoped" modules by path, packages under node_modules could only require each other and not packages from foobar/node_modules