Fork me on GitHub
#shadow-cljs
<
2023-04-26
>
Chris McCormick08:04:52

I am building a plugin for an app that runs inside Electron. I have a shadow-cljs project up and running using :target :browser for the plugin code and it's working great. However I have arrived at a situation where I need access to a library which uses node calls. These are available inside the Electron space - I can open up a console and call the node functions, but when I try to include this library shadow-cljs complains about these node libraries as it is targeting browser: The required JS dependency "node:stream" is not available, it was required by "node_modules/node-fetch/src/index.js". I tried building it as a :node-script and :node-library and they build but then there are errors with resolving the intermediate js files in dev mode:

Uncaught Error: ENOENT, services/.shadow-cljs/builds/plugin/dev/out/cljs-runtime/goog.debug.error.js not found in /tmp/.mount_joplinkRF3TG/resources/app.asar
I guess this is because it's running from inside an asar which maybe screws up shadow's path resolution? I also tried :js-options :resolve but I couldn't make that work either and in the end I had to have a lot of entries for all the node modules called. Anyway, what I really want is to build for browser but for shadow-cljs to ignore those node:* requires and jsut let them through as they will resolve correctly in the deployed environment (inside Electron). Any advice?

Chris McCormick08:04:31

I got around this previously when I had to use the fs module by doing this: (let [fs (js/require "fs")] ...) so the require happens at runtime. I could write a small .js file which does what I want and do this but I would prefer a cleaner solution.

thheller08:04:31

you maybe want to use :target :npm-module or :target :esm?

thheller08:04:01

or :js-options {:keep-as-require #{"node:stream"}}, :browser might not be ideal for electron since it bundles npm deps, which may lead to duplicates? but I guess build size is not that important

Chris McCormick09:04:59

Yeah size is not that important and I want it to bundle the npm deps because the plugin can't npm install anything.

Chris McCormick09:04:40

Thanks for the tip about :keep-as-require - will this flow through to included modules or it's only in the top level cljs code?

thheller09:04:07

all code shadow-cljs processes will leave it as is

๐Ÿ™ 2
Chris McCormick09:04:14

I have not tried :target :esm yet, will check that out. I tried :npm-module but I get the same file resolution issues:

Uncaught Error: Cannot find module './cljs_env'
Require stack:
- /tmp/.mount_joplinj3IwdH/resources/app.asar/services/plugins/plugin_index.html
    at Module._resolveFilename (node:internal/modules/cjs/loader:940:15)
    at i._resolveFilename (node:electron/js2c/renderer_init:33:1095)
    at Module._load (node:internal/modules/cjs/loader:785:27)
    at c._load (node:electron/js2c/asar_bundle:5:13343)
    at i._load (node:electron/js2c/renderer_init:33:356)
    at Module.require (node:internal/modules/cjs/loader:1012:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at plugin_com.joplinwebsite.publishplugin.js:1:13

thheller09:04:39

for :esm you'd need type="module" in the html

thheller09:04:05

but I'm not sure what the state of ESM for electron is. it wasn't supported last time I checked, dunno if that has changed

Chris McCormick09:04:15

Ah ok. I don't think I have control over how the app is requiring the plugin code. Will have to investigate if I can tell it it's a module (or if Electron even allows it as you point out).

Chris McCormick09:04:06

I have this target now using :keep-as-require:

:plugin {:target :browser
                   :output-dir "plugin"
                   :modules {:index {:entries [joplinsite.plugin]}}
                   :js-options {:keep-as-require #{"node:http"}}}
and I am seeing the following from shadow-cljs when compiling:
The required JS dependency "node:http" is not available, it was required by "node_modules/node-fetch/src/index.js".

Dependency Trace:
	joplinsite/plugin.cljs
	node_modules/node-fetch/src/index.js

Searched for npm packages in:
	/home/chrism/dev/joplinwebsite.com/node_modules

See: 
Did I make a mistake with the keep-as-require declaration?

thheller09:04:54

hmm no should be ok

thheller09:04:57

let me check

thheller09:04:17

ah, add :keep-native-requires true to the :js-options

thheller09:04:36

bad boolean logic couples it to that ๐Ÿ˜›

Chris McCormick09:04:32

Ok, that seems to have worked, thanks.

Chris McCormick09:04:00

I'm now seeing this:

[:plugin] Build failure:
Failed to inspect file
  /home/chrism/dev/joplinwebsite.com/node_modules/fetch-blob/index.js

it was required from
  /home/chrism/dev/joplinwebsite.com/node_modules/node-fetch/src/body.js

Errors encountered while trying to parse file
  /home/chrism/dev/joplinwebsite.com/node_modules/fetch-blob/index.js
  {:line 45, :column 2, :message "'}' expected"}
Will dig in and see if I can figure out where that is coming from.

thheller09:04:45

thats the closure compiler not recognizing too modern code

Chris McCormick09:04:09

Ah yes there is all kinds of es stuff in there.

thheller09:04:18

thats not a problem

thheller09:04:37

some non-standard cutting edge features just aren't supported yet

thheller09:04:28

assuming you are on the latest version already, then there isn't much to do unfortunately

thheller09:04:38

I'm pretty sure you can just keep all npm packages as they are

thheller09:04:51

and just bundle then as part of your package

thheller09:04:01

without shadow bundling them as part of the build

Chris McCormick09:04:17

Ok, this has got me a bit further and was very helpful, thanks!

Chris McCormick09:04:51

> and just bundle then as part of your package Ah I see what you mean, like link the node_modules in there and require them at runtime.

Chris McCormick09:04:41

Oh or pre-process them into a separate bundle.

thheller09:04:03

I don't know what the electron best practices are, but I assume that it doesn't require everything to be bundled

Chris McCormick12:04:52

Alright I finally got it working. I use shadow-cljs to generate an esm module with :keep-as-require for everything that wouldn't come in without errors, then I used webpack to bundle all the resulting esm code into something I can require dynamically at runtime in my plugin. It's a big hack but it seems to work (at least to the point where it's importing and I can run the function). Thanks for your help!

thheller15:04:48

might as well use webpack for everything then

๐Ÿ˜ฑ 2
Ryan18:04:52

Hey all, anyone with any clues of fixing the following:

web.js:1279 TypeError: Cldr.load is not a function
    at new WeekFields (index.js:14600:12)
    at WeekFields.ofFirstDayOfWeekMinDays (index.js:14580:17)
    at WeekFields.of (index.js:14554:27)
    at eval (index.js:14642:33)
    at eval (index.js:8:66)
    at shadow$provide.module$node_modules$$js_joda$locale_en_us$dist$index (index.js:7:1)
    at shadow.js.jsRequire (js.js:66:18)
    at shadow.js.require (js.js:113:20)
    at eval (tick.locale_en_us.js:2:68)
    at eval (<anonymous>)
I goofed something up .. Iโ€™m assuming I have the wrong version of jodatime installed in npm, but was just trying to see if anyone had any clues before I go stomping around trying to sort out how I got out of sorts. ๐Ÿ™‚ Prob. not shadow-cljs specific issue, but it seems somehow complected at the moment.

Ryan18:04:05

For anyone that stumbles across a similar problem: I removed all traces of joda time from my package.json and package-lock.json, had node uninstall them then re-started shadow-cljs which discovered the correct versions of the deps and installed them and now weโ€™re good again ๐Ÿ™‚

michaeldrogalis22:11:32

Hey, I've run smack into this. What do you mean "shadow-cljs discovered the correct versions of the deps"?