shadow-cljs

danielcompton 2025-07-28T01:32:58.944909Z

I tried to upgrade Shadow from 2.28.23 to 3.0.0 (and newer versions) and have found a bug/issue. We have a dependency on the NPM package exifreader . After upgrading we get a compile error:

The required JS dependency "https" is not available, it was required by "node_modules/exifreader/dist/exif-reader.js".
Minimal repro: https://github.com/danielcompton/shadow-exif-bug

danielcompton 2025-07-30T04:18:51.372189Z

After adding

:resolve {"https" false
                         "http" false
                         "fs" false
                         "path" false
                         "stream" false
                         "zlib" false}
I was able to get the app to compile, but now it crashes at runtime/test time with:
Chrome Headless 138.0.0.0 (Mac OS 10.15.7) WARN: 'shadow-cljs - failed to load', 'module$node_modules$contentstream$node_modules$readable_stream$lib$_stream_readable'

Chrome Headless 138.0.0.0 (Mac OS 10.15.7) ERROR: TypeError: Object prototype may only be an Object or null: undefined
TypeError: Object prototype may only be an Object or null: undefined
    at Object.create (<anonymous>)
    at module.exports [as inherits] ()
    at shadow$provide.module$node_modules$contentstream$node_modules$readable_stream$lib$_stream_readable ()
    at shadow.js.jsRequire ()
    at shadow$provide.module$node_modules$contentstream$node_modules$readable_stream$readable ()
    at shadow.js.jsRequire ()
    at shadow$provide.module$node_modules$contentstream$index ()
    at shadow.js.jsRequire ()
    at shadow$provide.module$node_modules$save_pixels_jpeg_js_upgrade$save_pixels ()
    at shadow.js.jsRequire ()
I think this code actually does want stream, how can I get the old Shadow behaviour of polyfilling node-only packages?

thheller 2025-07-30T06:49:08.169829Z

npm install node-libs-browser

thheller 2025-07-30T06:50:29.980999Z

but it may require some additional :resolve tweaks since the package names are often different

thheller 2025-07-30T06:50:56.814439Z

:resolve {"stream" {:target :npm :require "stream-browserify"}} and so on

danielcompton 2025-07-30T06:51:16.134789Z

Oh nice, thanks

thheller 2025-07-30T06:51:19.996939Z

did this really work in the browser before? seems to be using a lot of node stuff?

danielcompton 2025-07-30T06:51:43.936229Z

Yup, definitely worked :) I’m a little surprised too

danielcompton 2025-07-30T06:51:57.544189Z

Not sure if I’ll need all those libs

danielcompton 2025-07-30T06:52:03.483089Z

Hopefully just stream

danielcompton 2025-07-31T02:43:10.969209Z

It looks like stream is all that was needed for our browser build. We also build our app for Node.js, I’m having issues running it in Node:

file:///Users/.../renderer/src/renderer.js:17
export const $jscomp = {};
             ^

SyntaxError: Identifier '$jscomp' has already been declared
    at compileSourceTextModule (node:internal/modules/esm/utils:346:16)
    at ModuleLoader.moduleStrategy (node:internal/modules/esm/translators:146:18)
    at #translate (node:internal/modules/esm/loader:431:12)
    at ModuleLoader.loadAndTranslate (node:internal/modules/esm/loader:478:27)
    at async ModuleJob._link (node:internal/modules/esm/module_job:110:19)
When I comment out that line, then the next error I get is
ReferenceError: window is not defined
    at red.electron.electron_api (file:///.../renderer/src/renderer.js:15504:546)
    at file:///.../renderer/src/renderer:15504:615
    at ModuleJob.run (node:internal/modules/esm/module_job:263:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:540:24)
    at async init (file:///.../renderer/src/main.cjs:52:20)
Do I need to set some different polyfills for the Node build? Here is the relevant build config
:renderer
  {:target :esm
   :runtime :node
   ; ↓ required otherwise loader throws errors. See: 
   :closure-defines {shadow.loader.TEST true}
   :compiler-options {:optimizations :simple}
   :modules {:renderer {:exports {:init renderer.core/init
                }}}
   :output-dir "../renderer/src"
   :asset-path "/s/app"
   :css-options {:output-to "../renderer/src/bundle.css"}
   :build-hooks [(app.shadow-hooks/gen-output-report)
                 (red.css.core/hook)]
   :js-options {:keep-as-import
                #{"path" "jsdom" "canvas" "jsdom/lib/jsdom/living/generated/utils.js"
                  "fs" "https" "http" "@whimsicalcode/hunspell-wasm"
                  "@frontapp/plugin-sdk" "gif-frames"}}}}

 :build-defaults
 {:js-options {:hide-warnings-for
               #{"node_modules/ndarray/ndarray.js"}}
  :compiler-options {:externs ["externs/lib.ext.js"
                               "externs/paper.ext.js"
                               "externs/navigator.ext.js"
                               "externs/idb.ext.js"
                               "externs/svg.ext.js"
                               "externs/exifreader.ext.js"]
                     ;; keeping warnings as warning in dev for nicer
                     ;; reporting in a browser and clean errors in terminal
                     :warnings-as-errors false
                     ;; Don't break the build for deprecation warnings.
                     :warnings {:fn-deprecated false}
                     :output-feature-set :es2019}
  :release {:compiler-options {:warnings-as-errors true}}}

danielcompton 2025-07-28T01:33:22.212439Z

Looking at exifreader I can see that it is requiring “https” in a node-specific branch: https://github.com/mattiasw/ExifReader/blob/4ed84e8169a8b8fca3c95adbca4eb05b61beea29/src/exif-reader.js#L60-L74

danielcompton 2025-07-28T01:34:01.908369Z

Is there a way that I can tell Shadow to “provide” https without actually providing it?

danielcompton 2025-07-28T01:37:32.427159Z

I can open a bug in the Shadow-CLJS repo, but I’m not even sure if it is a Shadow bug or a ClojureScript bug, or just a misconfiguration

thheller 2025-07-28T07:45:14.262769Z

:js-options {:resolve {"https" false}}

thheller 2025-07-28T07:46:07.997569Z

in 3.x I removed the automatic polyfilling of node-only packages. exactly for cases such as this where you'd actually end up with an unused bloated dependency without noticing.

👍 1
❤️ 1
danielcompton 2025-07-28T08:22:11.657439Z

Nice! Is there a v3 upgrade guide? Want me to make a PR?

thheller 2025-07-28T08:25:18.460179Z

There is no guide since I didn't expect this to be an issue. Webpack moved away from automatic polyfilling 5+ years ago, so I assumed that the npm world had sorted this out by now and things would just work

thheller 2025-07-28T08:25:55.037969Z

but due to the occasional report that seems to have been naive 😛

thheller 2025-07-31T07:20:59.767049Z

ok a couple completely unrelated things it seems

thheller 2025-07-31T07:23:08.924759Z

window is not defined is you accessing js/window in some way. that indeed does not exist in node and never has. it was also never polyfilled? so shouldn't be a "new" problem?

thheller 2025-07-31T07:25:07.983319Z

export const $jscomp = {}; has been reported before. haven't figured that out one yet. kinda hard to reproduce, but I know the cause. just not why it happens in the first place

thheller 2025-07-31T07:27:04.561709Z

ah I see jsdom. maybe you used that to emulate window prior? if you didn't use :esm before that may be the problem. maybe that needs to be adjusted to export go globalThis?