Hi, I'm tracking down a npm dependency issue in my production app that I'm having trouble reproducing in a minimal demo. Pulling in the raven-js npm dependency in my prod app gives me a syntax error when requiring the module. I can trace it to an innocuous-looking function in a utilities js file. The only difference I can see is in the 'Sources' tab in the Chrome inspector: a file under node_modules/raven-js/src/utils.js exists in my test project, but in prod it's missing (all neighbouring source files are there). This might be a red herring but I can't see any other differences worthy of note.

The file module\$node_modules\$raven_js\$src\$utils.js is served up in both projects, along with the .map file.

Any guidance on things I could try greatly appreciated, as I feel a bit like I'm shooting in the dark!

@lsnape what is the error you get?

and you do not get this problem during development or is the code just not used in dev?

wait nvm. thats a dev error

hmm it compiles and loads fine for me

do you use lein?

Yea it's dev. I'm actually getting boot to serve the assets out of the target dir

what do you use to run shadow-cljs? just shadow-cljs or lein or boot?

It's a bit cobbled together atm. I run shadow-cljs watch app from one terminal window, and boot to serve assets out of target/public.

shadow-cljs standalone though? no lein, boot or tools.deps?

I'm chaining watch tasks so boot copies the compiled JS into the dir we serve.

trying to rule out dependency issues

Yeah standalone shadow.

ok

and boot copies everything correctly? not that it just stopped in the middle of the file or something?

check if the <output-dir>/cljs-runtime/module\$node_modules\$raven_js\$src\$utils.js file is corrupted or so

or rather check if the file generated by shadow-cljs is identical to the file served by boot

Doing that now. I'll diff it against the demo app, which is working.

There is a difference:

1,3c1,3
< shadow$provide.module$node_modules$raven_js$src$utils=function(global,process,require,module,exports){function isUndefined(what){return void 0===what}function isPlainObject(what){return"[object Object]"===Object.prototype.toString.call(what)}function isString(what){return"[object String]"===Object.prototype.toString.call(what)}function isArray(what){return"[object Array]"===Object.prototype.toString.call(what)}function supportsFetch(){if(!("fetch"in _window))return!1;try{return new Headers,new Request(""),
< new Response,!0}catch(e){return!1}}function each(obj,callback){var i,j;if(isUndefined(obj.length))for(i in obj)hasKey(obj,i)&&callback.call(null,i,obj[i]);else if(j=obj.length)for(i=0;i<j;i++)callback.call(null,i,obj[i])}function truncate(str,max){if("number"!==typeof max)throw Error("2nd argument to `truncate` function should be a number");return"string"!==typeof str||0===max?str:str.length<=max?str:str.substr(0,max)+"…"}function hasKey(object,key){return Object.prototype.hasOwnProperty.call(object,
< key)}function joinRegExp(patterns){for(var sources=[],i=0,len=patterns.length,pattern;i<len;i++)pattern=patterns[i],isString(pattern)?sources.push(pattern.replace(/([.*+?^=!:}()|\[\]\/\\])/g,"\\$1")):pattern&&pattern.source&&sources.push(pattern.source);return new RegExp(sources.join("|"),"i")function htmlElementAsString(elem){var out=[],className,attr;if(!elem||!elem.tagName)return"";out.push(elem.tagName.toLowerCase());elem.id&&out.push("#"+elem.id);if((className=elem.className)&&isString(className)){var classes=
---
> shadow$provide.module$node_modules$raven_js$src$utils=function(global,process,require,module,exports,shadow$shims){function isUndefined(what){return void 0===what}function isPlainObject(what){return"[object Object]"===Object.prototype.toString.call(what)}function isString(what){return"[object String]"===Object.prototype.toString.call(what)}function isArray(what){return"[object Array]"===Object.prototype.toString.call(what)}function supportsFetch(){if(!("fetch"in _window))return!1;try{return new Headers,
> new Request(""),new Response,!0}catch(e){return!1}}function each(obj,callback){var i,j;if(isUndefined(obj.length))for(i in obj)hasKey(obj,i)&&callback.call(null,i,obj[i]);else if(j=obj.length)for(i=0;i<j;i++)callback.call(null,i,obj[i])}function truncate(str,max){if("number"!==typeof max)throw Error("2nd argument to `truncate` function should be a number");return"string"!==typeof str||0===max?str:str.length<=max?str:str.substr(0,max)+"…"}function hasKey(object,key){return Object.prototype.hasOwnProperty.call(object,
> key)}function joinRegExp(patterns){for(var sources=[],i=0,len=patterns.length,pattern;i<len;i++)pattern=patterns[i],isString(pattern)?sources.push(pattern.replace(/([.*+?^=!:${}()|\[\]\/\\])/g,"\\$1")):pattern&&pattern.source&&sources.push(pattern.source);return new RegExp(sources.join("|"),"i")}function htmlElementAsString(elem){var out=[],className,attr;if(!elem||!elem.tagName)return"";out.push(elem.tagName.toLowerCase());elem.id&&out.push("#"+elem.id);if((className=elem.className)&&isString(className)){var classes=

That's diffing broken working

can't see anything 😛

first version is older though

First is using shadow 2.4.8.

might just be a closure compiler bug. try upgrading

OK, sorry should have checked that.

hmm no compiler changes since then though

otherwise try deleting the .shadow-cljs/builds directory just to rule out a cache issue

unlikely but never hurts to try 😉

OK will do

can't really tell what else is different in the diff besides ,shadow$shims which was added recently

other than that is appears similar. be sure to check the actual output served by boot though might be something happening in transit

yeah, eventually I'd like to isolate boot completely.

OK, still broken but the diff has changed:

3c3
< key)}function joinRegExp(patterns){for(var sources=[],i=0,len=patterns.length,pattern;i<len;i++)pattern=patterns[i],isString(pattern)?sources.push(pattern.replace(/([.*+?^=!:}()|\[\]\/\\])/g,"\\$1")):pattern&&pattern.source&&sources.push(pattern.source);return new RegExp(sources.join("|"),"i")function htmlElementAsString(elem){var out=[],className,attr;if(!elem||!elem.tagName)return"";out.push(elem.tagName.toLowerCase());elem.id&&out.push("#"+elem.id);if((className=elem.className)&&isString(className)){var classes=
---
> key)}function joinRegExp(patterns){for(var sources=[],i=0,len=patterns.length,pattern;i<len;i++)pattern=patterns[i],isString(pattern)?sources.push(pattern.replace(/([.*+?^=!:${}()|\[\]\/\\])/g,"\\$1")):pattern&&pattern.source&&sources.push(pattern.source);return new RegExp(sources.join("|"),"i")}function htmlElementAsString(elem){var out=[],className,attr;if(!elem||!elem.tagName)return"";out.push(elem.tagName.toLowerCase());elem.id&&out.push("#"+elem.id);if((className=elem.className)&&isString(className)){var classes=

replace(/([.*+?^=!:}()|\[\]\/\\])/g,"\\$1")) replace(/([.*+?^=!:${}()|\[\]\/\\])/g,"\\$1"))

Hmm this is actually looks like a raven-js lib version discrepancy.

Nope they are both the same 😞

@lsnape do you have anything in your config which might affect this? custom :js-options?

custom closure-compiler dependency?

I don't actually think that there is anything you do via the config to affect this but gotta rule it out still

@thheller I think I know what it is. We've got an asset fingerprinting boot task that scans all source files for template params ${param} and replaces them with a hashed version, or in the case of development skips over them. It looks like the regex is being incorrect replaced by the task. 😞

FWIW shadow-cljs has that built-in via https://shadow-cljs.github.io/docs/UsersGuide.html#NameHashing

It's an easy fix but very annoying. Thanks for working it through with me though.

ah I guess you mean images and stuff

Yeah we need it for all assets. Do you think that's out of scope in shadow?

no. I have plans for that. just not the time to implement it currently.

If shadow supported hashing all assets then we could ditch boot entirely!

Here's the boot task I wrote to take care of it: https://github.com/ELiTLtd/boot-asset-fingerprint

Feel free to steal anything out of there. It's something I'd be interesting in doing with a bit of guidance? Up to you!

so its just about change the references in the HTML?

I thought you were hashing images and stuff?

Yeah, it hashes the file names and the references in other files, such as html and css.

So ${img/foo.png} in bar.html becomes img/foo-af28b.png in the html file, plus a new file is written to the file path.

yeah. the example only had .js so I thought it was just about .js

I really like the way you do it by generating a manifest file for module hashing. It's more work to plumb it in the project, but it's very flexible.

yeah in my work projects I just watch the manifest for changes in production

so I can deploy new JS code without restarting the server

but its a clojure server generating the HTML anyways so no static html to rewrite

That's what we're doing in our latest project: html page frame is all hiccup rendered on request (memoized).

But older projects serve up html templates rendered after cljs compilation time.

I'm still struggling with shadow-cljs in CI. running a release build takes up to 10 mins

did you enable caching?

yep. persisting the .shadow-cljs/builds/ folder shaved off about a minute or so

did you run with --verbose? cache should be way more than that

verbose should tell you exactly where the time is spent

OK, I'll dig into it more

for reference, about half of that is just starting shadow-cljs 😬 compiling takes about 5 mins

5min to start? are you running on a toaster?

😂

very well could be for all I know

I'm trying to compile a target :karma build:

-> build target: :karma stage: :configure
NullPointerException:
	shadow.build.targets.karma/configure (karma.clj:21)
	shadow.build.targets.karma/configure (karma.clj:16)
	shadow.build.targets.karma/process (karma.clj:112)
	shadow.build.targets.karma/process (karma.clj:108)
	clojure.lang.Var.invoke (Var.java:381)
	shadow.build/process-stage/fn--15189 (build.clj:120)
	shadow.build/process-stage (build.clj:117)
	shadow.build/process-stage (build.clj:109)

I'm a little unsure if my regexp is working. would that trigger this error?

missing :output-to, need to add specs for the config still

ahh, I had output-dir

I'm getting an error when trying to run karma tests:

d on socket wpx59euvS82blWCjAAAA with id 10464947
HeadlessChrome 0.0.0 (Mac OS X 10.13.5) ERROR
  {
    "message": "Uncaught ReferenceError: shadow is not defined\nat : shadow is not defined\n    at eval (eval at <anonymous> (/Users/r627543/Code/fido/packages/web/app/node_modules/karma-cljs-test/adapter.js:6:7), <anonymous>:1:1)\n    at ContextKarma.start (/Users/r627543/Code/fido/packages/web/app/node_modules/karma-cljs-test/adapter.js:6:7)\n    at ContextKarma.loaded ()\n    at ",
    "str": "Uncaught ReferenceError: shadow is not defined\nat : shadow is not defined\n    at eval (eval at <anonymous> (/Users/r627543/Code/fido/packages/web/app/node_modules/karma-cljs-test/adapter.js:6:7), <anonymous>:1:1)\n    at ContextKarma.start (/Users/r627543/Code/fido/packages/web/app/node_modules/karma-cljs-test/adapter.js:6:7)\n    at ContextKarma.loaded ()\n    at "
  }

I'm guessing that karma isn't loading the correct js?

looks like its trying to call the shadow.test.karma.init fn maybe and doesnt find it?

this is what my config looks like:

{:target :karma
           :ns-regexp ".devcards$"
           :output-to "target/karma/tests.js"}

and I copied the karma.conf.js from the user-guide

and you changed the path there?

module.exports = function (config) {
  config.set({
    browsers: ['ChromeHeadless'],
    // The directory where the output file lives
    basePath: 'target/karma/',
    // The file itself
    files: ['target/karma/tests.js'],
    frameworks: ['cljs-test'],
    plugins: ['karma-cljs-test', 'karma-chrome-launcher'],
    colors: true,
    logLevel: config.LOG_INFO,
    client: {
      args: ["shadow.test.karma.init"],
      singleRun: true
    }
  })
};

nvm I figured it out

files is relational to basePath

basePath + files doesn't work I think

yep

😅 PEBKAC. thanks for holding my hand