ah right. I only tested with foo.html which automatically sets content-type based on file ext 😛

fixed in [email protected]

haha … I had an idea for externs yesterday and it might actually work 🙂

probably a really bad idea but I’m gonna test the size difference on my work project to check how bad

cool!

one question, how do transitive JS deps work? eg. I have a commands library that will depend on keypress.js as a github dependency in package.json, and am wondering what I do in that library so that I can depend on it elsewhere

yeah thats unsolved at the moment

CLJS wants to do it this way: deps.cljs in your src root, {:npm-deps {"name-of-dep" "version"}]

shadow-cljs npm-deps will attempt to install them but that needs work, I’m not happy with how that works

I’m sort of leaning towards including a package.json in your .jar

but deps.cljs is probably the way to go for now

ok, fair enough

Ever seen something like this? cljs.core.async is compiling with a trailing js.map:

cljs.core.async.partition_by.cljs$lang$maxFixedArity = 3;


//# sourceMappingURL=cljs.core.async.js.map
js.map

(leading to Uncaught ReferenceError: js is not defined)

maybe 2 shadow-cljs watches running?

no clue otherwise

am watching 3 builds at once

with one watch command unless there is something running i’m not seeing

is there a wiki page or sth where i should make notes of these JS related Q&A’s / stubs for documentation?

i’ll just start a page in the shadow-eval repo wiki

for now just my own notes to remember: https://github.com/mhuebert/shadow-eval/wiki

yeah I haven’t written down anything yet

nice start

haha this externs stuff was waaaaaay easier to implement than I expected

looking good so far

haha it works

excellent!

that should cover almost all externs … as long as you dont use :resolve :target :global

and some dynamic programming things jquery and others do

what I do is so simple .. for every JS dependency shadow-cljs imports

it collects all assign property names, and creates externs for them

exports.foo = 1; would collect the foo

exports.thing = {a: 1}; would collect thing and a

thats all

that sounds great, i look forward to trying it

you parse the js AST for this?

yes

but I do this anyways to process the require in JS files

https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/build/closure/ReplaceRequirePass.java#L22-L52

quite simple actually

working at the right depth in the stack / layer of abstraction can make things so simple

need to work out how to do this for node but for the browser it seems to actually work reasonably well

still doing a few tests to confirm

/* @constructor */ function ShadowJS() {};
ShadowJS.prototype.$$typeof;
ShadowJS.prototype.Children;
ShadowJS.prototype.Component;
ShadowJS.prototype.PureComponent;
ShadowJS.prototype.ReactCurrentOwner;
ShadowJS.prototype.__SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED;
ShadowJS.prototype.__self;
ShadowJS.prototype.__source;
ShadowJS.prototype._owner;
ShadowJS.prototype.assign;
ShadowJS.prototype.children;
ShadowJS.prototype.cloneAndReplaceKey;
ShadowJS.prototype.cloneElement;
ShadowJS.prototype.constructor;
ShadowJS.prototype.context;
ShadowJS.prototype.count;
ShadowJS.prototype.createElement;
ShadowJS.prototype.createFactory;
ShadowJS.prototype.current;
ShadowJS.prototype.enqueueForceUpdate;
ShadowJS.prototype.enqueueReplaceState;
ShadowJS.prototype.enqueueSetState;
ShadowJS.prototype.forEach;
ShadowJS.prototype.forceUpdate;
ShadowJS.prototype.framesToPop;
ShadowJS.prototype.func;
ShadowJS.prototype.isMounted;
ShadowJS.prototype.isPureReactComponent;
ShadowJS.prototype.isReactComponent;
ShadowJS.prototype.isValidElement;
ShadowJS.prototype.key;
ShadowJS.prototype.keyPrefix;
ShadowJS.prototype.map;
ShadowJS.prototype.name;
ShadowJS.prototype.only;
ShadowJS.prototype.props;
ShadowJS.prototype.prototype;
ShadowJS.prototype.ref;
ShadowJS.prototype.refs;
ShadowJS.prototype.render;
ShadowJS.prototype.result;
ShadowJS.prototype.setState;
ShadowJS.prototype.thatReturns;
ShadowJS.prototype.thatReturnsArgument;
ShadowJS.prototype.thatReturnsFalse;
ShadowJS.prototype.thatReturnsNull;
ShadowJS.prototype.thatReturnsThis;
ShadowJS.prototype.thatReturnsTrue;
ShadowJS.prototype.toArray;
ShadowJS.prototype.type;
ShadowJS.prototype.unstable_AsyncComponent;
ShadowJS.prototype.unstable_isAsyncReactComponent;
ShadowJS.prototype.updater;
ShadowJS.prototype.version;

doesn’t look all that bad, thats just React v16 and its deps

crap its missing the lifecycle fns 😞

nevermind those are part of react-dom

woho

;; auto
Flushing: base.js (181706 bytes)
Flushing: react.js (111923 bytes)
Flushing: demo.js (195720 bytes)
Flushing: extra.js (136 bytes)
Flushing: worker.js (75 bytes)

;; manual
Flushing: base.js (181233 bytes)
Flushing: react.js (111923 bytes)
Flushing: demo.js (195684 bytes)
Flushing: extra.js (136 bytes)
Flushing: worker.js (75 bytes)

not at all bad, auto is using the generated externs, manual is using the default react externs

pretty much no diff

I call this a success

@mhuebert [email protected] has this enabled by default, if you feel like testing.

I’m gonna verify this on my work project now

cool, just in the middle of another thing atm but can try later

that is remarkably similar

I can’t believe how fast these advanced builds are

decided to try importing firebase, eg. yarn install firebase and then ["firebase/app" :as firebase]. it is supposed to work with webpack etc.; I get:

ExceptionInfo: no source by name: node_modules/@firebase/util/dist/cjs/src/constants.ts

not sure why it would be looking for constants.ts vs constants.js, which does exist

React appears to work now without supplying any externs manually 🙂

adding something to :npm-deps in deps.cljs does not appear to have any effect, with or without :lein true

@mhuebert yes :npm-deps is not finished. you can try shadow-cljs npm-deps but it has some issues

ok.

until thats sorted out you need to trigger the install manually

does shadow look in the node_modules of dependencies?

hmm?

so Maria depends on Commands which has an npm dependency keypress.js

does keypress.js need to be in the node_modules of Maria, or can it be in Commands

I don’t understand

{:npm-deps {"keypress.js" "<version>"}} is in the deps.cljs of the commands project

it is only a transitive dep of Maria.

does it somehow need to end up being installed in maria/node_modules or would it also be picked up in commands/node_modules

shadow-cljs ONLY checks <project-root>/node_modules/.... never anywhere else

ok

so ultimately shadow-cljs npm-deps could crawl other projects’ deps.cljs and install all the listed :npm-deps into the current project

thats what it does yes

makes sense.

looking at the firebase error

haha

doh 🙂

its a bit too smart

the firebase files have source maps, which apparently causes it to return constants.ts instead of constants.js when asking which filename it is processing currently

thats awesome and scary 😛

SEVERE: node_modules/@firebase/util/dist/cjs/src/constants.js:26:
Originally at:
node_modules/@firebase/util/dist/cjs/src/constants.ts:26: ERROR - @define variable  assignment must be global

Oct 13, 2017 3:28:12 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: node_modules/@firebase/util/dist/cjs/src/constants.js:30:
Originally at:
node_modules/@firebase/util/dist/cjs/src/constants.ts:31: ERROR - @define variable  assignment must be global

this is the problem though

@define is a closure annotation, will disable that check if possible

@mhuebert should be fixed in [email protected]

can you please check the line count of target/shadow-cljs/builds/browser/release/shadow.js.auto_externs.js?

total curious how many that generates for your project

deleted target dir, still seem to be getting the same error with firebase

with 2.0.15

no source by name: node_modules/@firebase/util/dist/cjs/src/constants.ts
{:name "node_modules/@firebase/util/dist/cjs/src/constants.ts"}
ExceptionInfo: no source by name: node_modules/@firebase/util/dist/cjs/src/constants.ts

with

["firebase/app" :as firebase]
["firebase/auth"]

now.. wait

I had :lein true and had not updated the shadow-cljs version in project.clj

only in package.json

trying again

🙂

it still says 2.0.15 when loading the script, this could be confusing, but maybe there is no way to easily catch if the version is different in project.clj

1569 lines in auto_externs

this is just the ‘outer’ frame so not so many deps, firebase + react

lots of minified stuff in here

thats odd

build went from 400 > 600 kb

damn 😛

🙂

wonder where those minified props come from

can you send me the target/shadow-cljs/builds/browser/release/shadow-js/* files zipped or so?

there is probably a closure minified thing in there, want to see if I can somehow detect that

could it not be react, minified?

no, JS deps usually do not minify property assigns

only locals

https://github.com/firebase/firebase-js-sdk/blob/master/packages/auth/src/auth.js

weird, there is only the minified source available for this

but yeah that looks like it was minified by closure

no fun

those things will need externs

your build probably gets so large because all the short names are taken

the externs file “claims” all the a, b, c, Aa, Ab, etc names

so probably all vars in your build become 3 letter names instead of 2

that adds up quick

yeah exactly

it’s not a problem here, since firebase is a google thing they do maintain externs

provided externs can be looked up directly from node_modules

yeah but how do I protect the generator from extracting these

oh i see.

only thing I can think of is setting a “cap”

so if a namespace has too many properties you get a warning or something

and then add an exclude option to ignore properties from that file

no normal file should export this many props

yeah firebase has often only released minified stuff

or I just ignore the pattern of names

ah better idea

is there a way to disable this temporarily

i’d like to compare the gzipped sizes of with / without

:js-options {:generate-externs false}

i misspoke

sorry. it makes only a tiny difference in file size.

it still seems wrong 🙂

I have a quick fix

(set/superset? properties #{"a" "b" "c" "Aa" "Ab" "Ac"})

if the file contains those properties, I strip out every 1/2 letter property

475 properties for firebase/app + firebase/auth after

so over 1k removed. seems good.

so, an example to try

in a namespace with this required:

["firebase/app" :as firebase]
["firebase/auth"]

hmm?

for me it mangled auth

right. nothing I can do about that

the closure minified files use goog.exportSymbol calls to create the names

they look like this Z(a,"EmailAuthProvider",Ig,[]);

that is not a property assign

meh I might scrap this whole idea

CLJS already has a half-way working :infer-externs .. should just use that

i think whatever they have + a simple way to add additional properties might be decent

yeah

this strikes me as extremely hard to get right

the automation part is hard yes, thats why I was surprised why the generation thing worked so well

its still pretty good, just didn’t anticipate this 😛

I’ll look at :infer-externs more, maybe I can combine that in some way

the only part I didn’t like about :infer-externs was that you sometimes have to annotate your code

and often it would complain about things

hmm. could it make sense that firebase’s externs don’t work when firebase is required via npm vs. included in a script tag

is the firebase source available at all?

oh you have the firebase externs included and they don’t work?

they were recently open sourced: https://github.com/firebase/firebase-js-sdk

appears that way, checking again

but there is no source in there

only compiled things

oh right .. typescript

nevermind 😉

i wouldn’t worry too much about firebase in particular. i just thought it might be a good one to try because google usually annotates everything etc… but i guess that is more relevant for things handled by closure compiler

removing all the short names should fix most issues

i guess externs on short names wouldn’t make a big file impact, because you can’t rename these things shorter anyway? its not preventing the closure compiler from reducing the size of $some_long_name$

but instead of turning some_long_name into a its forced to turn it into abc or whatever

2 extra bytes per name is quite a lot

but thats fixed now

did you try with the firebase externs?

if they don’t work thats a problem

have to do a bit of code shuffling to try that properly

one sec

where are those externs?

node_modules/firebase/externs

no it works

i had to move some initialization code into cljs to test properly

ok good

I’ll test a few things with :infer-externs, maybe I can find a solution that is more reliable

should have done that in the first place, just got too excited by this idea of collecting all properties 😛

I mean it’s a pretty big win that a library like React ‘just works’ with this generate-externs option

yes but I think :infer-externs can cover this as well

and again after testing properly it does not appear to inflate sizes a lot

with lots of type hints

I used to be more strict about externs

too obsessed with file sizes over convenience

but should focus more on the user and :externs come up pretty much always

so generated something good, then let user optimize when needed is better

yeah

with React + Firebase, difference is only 2kb adding :generate-externs in addition to the ‘real’ externs for both libs

thats reasonable

generated externs appear to totally solve ‘React’ but fail with firebase due to auth issue mentioned above

its a tiny percentage, 601 vs 603 kb

:infer-externs should cover that

if you get a chance you could try master which has the naming fixes

but 2kb is ok enough already, probably doesn’t change much

0.3%, negligible

but should try it on some more files

I agree this is an important thing though, and worth the effort. has been one of the most frustrating aspects of ClojureScript, this gulf between dev and production where suddenly new bugs appear, and extremely slow feedback cycles with advanced compilation. (well: i am very surprised/impressed at how much faster it is with shadow. i thought the slowness was somehow necessary part of the process)

well shadow-cljs caches more aggressively so the only thing you pay for is the actual :advanced compilation

other CLJS tools do not cache at all for :advanced builds, so it always recompiles all CLJS as well

which typically takes longer than the optimize part

geez forgot how complicated this :infer-externs stuff is …

(js/console.log (firebase/auth)) got this part figured out

(js/console.log (.auth firebase)) is also valid but harder to track

small thing, if we could make the debugging statements in shadow.cljs.bootstrap.browser optional 🙂

@mhuebert oops yes, forgot to disable them

@jiyinyiyong those things don’t really apply to shadow-cljs 😉