I tried to upgrade Shadow from 2.28.23 to 3.0.0 (and newer versions) and have found a bug/issue. We have a dependency on the NPM package exifreader . After upgrading we get a compile error:
The required JS dependency "https" is not available, it was required by "node_modules/exifreader/dist/exif-reader.js".
Minimal repro: https://github.com/danielcompton/shadow-exif-bugAfter adding
:resolve {"https" false
"http" false
"fs" false
"path" false
"stream" false
"zlib" false}
I was able to get the app to compile, but now it crashes at runtime/test time with:
Chrome Headless 138.0.0.0 (Mac OS 10.15.7) WARN: 'shadow-cljs - failed to load', 'module$node_modules$contentstream$node_modules$readable_stream$lib$_stream_readable'
Chrome Headless 138.0.0.0 (Mac OS 10.15.7) ERROR: TypeError: Object prototype may only be an Object or null: undefined
TypeError: Object prototype may only be an Object or null: undefined
at Object.create (<anonymous>)
at module.exports [as inherits] ()
at shadow$provide.module$node_modules$contentstream$node_modules$readable_stream$lib$_stream_readable ()
at shadow.js.jsRequire ()
at shadow$provide.module$node_modules$contentstream$node_modules$readable_stream$readable ()
at shadow.js.jsRequire ()
at shadow$provide.module$node_modules$contentstream$index ()
at shadow.js.jsRequire ()
at shadow$provide.module$node_modules$save_pixels_jpeg_js_upgrade$save_pixels ()
at shadow.js.jsRequire ()
I think this code actually does want stream, how can I get the old Shadow behaviour of polyfilling node-only packages?npm install node-libs-browser
but it may require some additional :resolve tweaks since the package names are often different
:resolve {"stream" {:target :npm :require "stream-browserify"}} and so on
Oh nice, thanks
did this really work in the browser before? seems to be using a lot of node stuff?
Yup, definitely worked :) I’m a little surprised too
Not sure if I’ll need all those libs
Hopefully just stream
It looks like stream is all that was needed for our browser build. We also build our app for Node.js, I’m having issues running it in Node:
file:///Users/.../renderer/src/renderer.js:17
export const $jscomp = {};
^
SyntaxError: Identifier '$jscomp' has already been declared
at compileSourceTextModule (node:internal/modules/esm/utils:346:16)
at ModuleLoader.moduleStrategy (node:internal/modules/esm/translators:146:18)
at #translate (node:internal/modules/esm/loader:431:12)
at ModuleLoader.loadAndTranslate (node:internal/modules/esm/loader:478:27)
at async ModuleJob._link (node:internal/modules/esm/module_job:110:19)
When I comment out that line, then the next error I get is
ReferenceError: window is not defined
at red.electron.electron_api (file:///.../renderer/src/renderer.js:15504:546)
at file:///.../renderer/src/renderer:15504:615
at ModuleJob.run (node:internal/modules/esm/module_job:263:25)
at async ModuleLoader.import (node:internal/modules/esm/loader:540:24)
at async init (file:///.../renderer/src/main.cjs:52:20)
Do I need to set some different polyfills for the Node build?
Here is the relevant build config
:renderer
{:target :esm
:runtime :node
; ↓ required otherwise loader throws errors. See:
:closure-defines {shadow.loader.TEST true}
:compiler-options {:optimizations :simple}
:modules {:renderer {:exports {:init renderer.core/init
}}}
:output-dir "../renderer/src"
:asset-path "/s/app"
:css-options {:output-to "../renderer/src/bundle.css"}
:build-hooks [(app.shadow-hooks/gen-output-report)
(red.css.core/hook)]
:js-options {:keep-as-import
#{"path" "jsdom" "canvas" "jsdom/lib/jsdom/living/generated/utils.js"
"fs" "https" "http" "@whimsicalcode/hunspell-wasm"
"@frontapp/plugin-sdk" "gif-frames"}}}}
:build-defaults
{:js-options {:hide-warnings-for
#{"node_modules/ndarray/ndarray.js"}}
:compiler-options {:externs ["externs/lib.ext.js"
"externs/paper.ext.js"
"externs/navigator.ext.js"
"externs/idb.ext.js"
"externs/svg.ext.js"
"externs/exifreader.ext.js"]
;; keeping warnings as warning in dev for nicer
;; reporting in a browser and clean errors in terminal
:warnings-as-errors false
;; Don't break the build for deprecation warnings.
:warnings {:fn-deprecated false}
:output-feature-set :es2019}
:release {:compiler-options {:warnings-as-errors true}}} Looking at exifreader I can see that it is requiring “https” in a node-specific branch: https://github.com/mattiasw/ExifReader/blob/4ed84e8169a8b8fca3c95adbca4eb05b61beea29/src/exif-reader.js#L60-L74
Is there a way that I can tell Shadow to “provide” https without actually providing it?
I can open a bug in the Shadow-CLJS repo, but I’m not even sure if it is a Shadow bug or a ClojureScript bug, or just a misconfiguration
:js-options {:resolve {"https" false}}
in 3.x I removed the automatic polyfilling of node-only packages. exactly for cases such as this where you'd actually end up with an unused bloated dependency without noticing.
Nice! Is there a v3 upgrade guide? Want me to make a PR?
There is no guide since I didn't expect this to be an issue. Webpack moved away from automatic polyfilling 5+ years ago, so I assumed that the npm world had sorted this out by now and things would just work
but due to the occasional report that seems to have been naive 😛
ok a couple completely unrelated things it seems
window is not defined is you accessing js/window in some way. that indeed does not exist in node and never has. it was also never polyfilled? so shouldn't be a "new" problem?
export const $jscomp = {}; has been reported before. haven't figured that out one yet. kinda hard to reproduce, but I know the cause. just not why it happens in the first place
ah I see jsdom. maybe you used that to emulate window prior? if you didn't use :esm before that may be the problem. maybe that needs to be adjusted to export go globalThis?