Fork me on GitHub
#shadow-cljs
<
2023-10-30
>
diego.videco03:10:59

Hey all, I am trying to build an npm library to be consumed on the browser. I am confused by this statement on the docs (section https://shadow-cljs.github.io/docs/UsersGuide.html#target-npm-module`:npm-module`https://shadow-cljs.github.io/docs/UsersGuide.html#target-npm-module):

If you plan to distribute code on NPM, then you may want to use the :node-library target instead since it allows for a finer level of control over exports and optimization.
I am trying said :node-library option as I liked that it could bundle everything into a single file (couldn’t find a way with the :npm-module target). The bundle works on node, but on the browser I get an error Uncaught ReferenceError: global is not defined . Am I missing some option on the :node-library . Of course it seems like that option is just meant for node, but the statement in the aforementioned section seems to imply otherwise, and I did like the facilities given by the :node-library option.

thheller07:10:41

as the name should imply the :node-library is targetting a node runtime, therefore it will not work in the browser without additional processing

thheller07:10:35

how do you intend browsers to consume this? I mean how would they access the exports? regular JS doesn't have anything like that

thheller07:10:24

that'll work fine in node and the browser using <script type=module ...>

diego.videco14:10:35

Actually, the library is intended to be consumed by a Purescript app. I don’t know much about it, except that it is using esbuild at some in it’s build process. I think Purescript, with imports, so perhaps that would work.

diego.videco14:10:09

Wondering… the problem seems to only be the reference to the global object. I believe globalThis is the environment compatible reference. Is there a way to fix that?

thheller14:10:36

but I believe you don't have any actual JS dependencies in your build, so the :import part doesn't matter

thheller14:10:57

:esm is the better choice, as :node-library has very many assumptions about actually running in node

thheller14:10:15

:npm-module would also work

diego.videco15:10:29

Ok, I’ll try those. What’s the best way to configure the build so that I can export the api from a single file? Like this: https://github.com/diegovdc/erv/blob/main/src/js/export_fn.cljs#L9-L14 So that accessing the functions could be done like: erv.cps.make(...)

thheller15:10:52

ESM is too strict to make it that dynamic, so would need to be done in the build config

thheller15:10:19

I'd recommend to just create a defn that handles the translation between JS/CLJS

diego.videco03:10:55

The code is open source so here it is: https://github.com/diegovdc/erv/blob/main/src/js/export_fn.cljs https://github.com/diegovdc/erv/blob/main/shadow-cljs.edn Any suggestions are welcome. Ideally I would like to keep a single file for the bundle and an API like the one in export_fn.cljs.

Stefan10:10:16

Good morning! Every now and then run into a commonjs/esm/whathaveyou issue. Unfortunately my skull has proven too thick so far for me to understand exactly what's going on and why things fail now and then. Today it's react-tabulator. When I try to use it (`(:require ["react-tabulator" :refer [ReactTabulator]])`) I get an error in the browser: "Unhandled Promise Rejection: TypeError: undefined is not a constructor (evaluating 'new tabulator_tables_1.TabulatorFull')". When looking online I find people with similar issues who had to switch from using require to using import in JavaScript. I also found this: https://github.com/ngduc/react-tabulator/issues/215#issuecomment-917574928. My shadow-cljs project has :target :browser and a few :modules in shadow-cljs.edn, including one shared module. Can someone point me in the right direction? Thanks!!!

thheller12:10:19

hard to say. could just be a version conflict.

thheller12:10:30

if you want to setup a repro I can take a look

Stefan13:10:12

Thanks Thomas, I'll try to do that then.

Stefan07:10:33

I created a small repo that exhibits the same problem on my machine: https://github.com/svdo/shadowtabulator. Thanks for offering to look into it!

thheller07:10:44

ok yeah, this is due the tabulator-tables shipping two different dist versions. one for ESM. one for CommonJS, with an incompatible API (i.e. different export style)

thheller07:10:58

so react-tabulator imports it the "wrong" way and breaks

Stefan07:10:27

You gotta love it 🙂

thheller07:10:50

you can "fix" this by setting :js-options {:entry-keys ["module" "browser" "main"]} in the build config

thheller07:10:05

but be aware that this may break other packages

thheller07:10:17

commonjs/esm interop is still an absolute mess and not getting better

thheller07:10:41

works in the repro but doesn't mean it will in bigger projects with more dependencies

Stefan07:10:53

I'm not sure if it was that specific order that I tried yesterday, but that broke tons of things. But I'll try again 🙂

thheller08:10:01

you can try https://shadow-cljs.github.io/docs/UsersGuide.html#js-provider-external and see if that works. webpack uses module entries by default

thheller08:10:36

if it doesn't work also try adding :export-conditions ["import" "module" "browser" "require" "default"]

Stefan08:10:43

At the very least 'emotion' breaks with that js-options setting

thheller08:10:59

unfortunately the JS world keeps adding to this mess instead of fixing it 😛

thheller08:10:19

:export-conditions also goes in :js-options

👍 1
Stefan08:10:02

I can just put that in the dev section for trying it right?

:dev        {:js-options {:entry-keys ["module" "browser" "main"]
                             :export-conditions ["import" "module" "browser" "require" "default"]}
                :compiler-options {:output-feature-set :es8
                                   :infer-externs      :auto
                                   :external-config {:guardrails {}}
                                   :warnings {:fn-deprecated false}
                                   :closure-defines {goog.DEBUG true}}}
The entry-keys causes errors and the export-conditions doesn't seem to make a difference.

thheller08:10:33

then your release builds will just break, so not recommended in :dev

Stefan08:10:55

Yeah right but just to see if it works, which it doesn't apparently.

Stefan08:10:02

Switching to a js bundler feels like a step back. What's the state of the are there, what is the current simple+popular choice? Last time I used something like that it was rollup iirc?

thheller08:10:03

no clue, I never used it myself. I tend to not use many npm packages myself, so never run into those that don't work 😛

Stefan08:10:32

That's also a solution 😉

Stefan08:10:02

Anyway, thanks for clarifying and looking into it!!!

thheller08:10:24

good to know it works

Stefan08:10:58

what works? Not using it? 😛

thheller08:10:59

well I'd try webpack, thats most widely used and fixes most quirks

thheller08:10:35

but there are things even webpack can't fix. let me know the result if you try

Stefan08:10:24

I'm getting cold sweat from the memories of fighting webpack. I'll skip that for now and use Tabulator directly without react-tabulator.

andrea15:10:39

Does anyone have a working example of using a service worker? anything that looks strikingly wrong with this :bulids entry?

:service-worker {:target :browser
                           :output-dir "resources/public/assets"
                           :asset-path "/assets"
                           :modules {:service-worker {:entries [app.serviceworker.core]}}
                           :devtools {:after-load app.serviceworker.core/main}
                           :service-worker true}

thheller15:10:49

:service-worker true is not a valid option, you want :web-worker true

thheller15:10:26

otherwise looks fine

thheller15:10:36

but I believe service workers still have the restriction of needing to be in the root folder

thheller15:10:59

so your :output-dir likely needs to be resources/public and :asset-path "/" accordingly

andrea15:10:01

I think I'm passing the correct header when serving Service-Worker-Allowed: / And I am getting the correct file when I try to register, but it fails to execute the actual output script.

Uncaught Error: browser bootstrap used in incorrect target
  at service-worker.js:1253:11
    at service-worker.js:1394:3

thheller15:10:24

thats because of not using :web-worker true

thheller15:10:17

you can also use :target :esm btw I believe

andrea15:10:21

mmh I thought I changed that, I need to sort my caching I guess. One more thing I now noticed is that in the docs it's as module instead of at top-level within :builds I'll hack some more, thanks @U05224H0W

Lone Ranger22:11:07

@U47V0EZF1 if you figure out how to get a repl going with a service worker lemme know

andrea16:11:11

@U3BALC2HH I did get a service worker start but I'm fighting with the repl, I could share my messed up shadowcljs file if you want. The problem I have now is that if the service worker is running calva puts the repl inside the service worker rather than the core, I'm not sure how to deal with that yet

andrea16:11:46

To be honest I don't know how much of a shadowcljs or calva problem this is

andrea16:11:25

happy to hack in pair on an empty shadowcljs project just to produce a working template as I'm finding very little online

thheller17:11:22

whats the problem?

andrea17:11:37

I'm struggling to have a setup that works with re-frame-10x hotreload and a working repl in calva when serviceworkers are installed. In particular if I run (js/console.log js/self) from the repl I get different results depending on whether the serviceworker is installed and running at page load (ServiceWorkerGlobalScope) or not (Window). This seems not related to which file I'm on, ie: looks like the same repl. I've uploaded my edn file https://pastebin.com/hbsEMMqQ if that makes it any easier

thheller17:11:36

that indeed seems to be full of stuff that doesn't do anything 😛

😅 1
thheller17:11:46

I'll get back to you, busy currently

thheller18:11:34

{:deps {:aliases [:cljs]}
 :builds
 {:app
  {:target :browser
   :output-dir "resources/public/assets/js"
   :asset-path "/assets/js"
   :modules
   {:shared
    {:entries []}
    :service-worker
    {:depends-on #{:shared}
     :entries [com.mydomain.serviceworker.core]
     :web-worker true}
    :main
    {:init-fn com.mydomain.core/init
     :depends-on #{:shared}
     :preloads [day8.re-frame-10x.preload]}}


   :tailwind/output "resources/public/assets/css/site.css"
   :tailwind/config {:dark-mode "media"
                     :content ["src/cljs/**/*.cljs"]}

   :devtools
   {:reload-strategy :full
    :devtools-url ""}

   :build-options {:manifest-name "../../../manifest.edn"}
   :dev {:build-hooks [(teknql.tailwind/start-watch!)]
         :compiler-options
         {:closure-defines {re-frame.trace.trace-enabled? true
                            day8.re-frame.tracing.trace-enabled? true}
          :elide-asserts false}}

   :release {:build-options {:ns-aliases {day8.re-frame.tracing day8.re-frame.tracing-stubs}}
             :build-hooks [(teknql.tailwind/compile-release!)]}}}
 
 :dev-http {5003 "resources/public"}}

thheller18:11:46

ok this is with all the stuff removed that didn't do anything in the first place

thheller18:11:02

the way this works is that the REPL connects for each new runtime

thheller18:11:32

the browser itself is a runtime, and the service worker is another one. so 2 runtimes per build, and I'm not sure how you'd select which one you want to talk to in calva

thheller18:11:48

what might be easier to settle this is just using two builds separately

thheller18:11:13

so something like

{:deps {:aliases [:cljs]}
 :builds
 {:app
  {:target :browser
   :output-dir "resources/public/assets/js"
   :asset-path "/assets/js"
   :modules
   {:main
    {:init-fn com.mydomain.core/init
     :preloads [day8.re-frame-10x.preload]}}

   :tailwind/output "resources/public/assets/css/site.css"
   :tailwind/config {:dark-mode "media"
                     :content ["src/cljs/**/*.cljs"]}

   :devtools
   {:reload-strategy :full
    :devtools-url ""}

   :build-options {:manifest-name "../../../manifest.edn"}
   :dev {:build-hooks [(teknql.tailwind/start-watch!)]
         :compiler-options
         {:closure-defines {re-frame.trace.trace-enabled? true
                            day8.re-frame.tracing.trace-enabled? true}
          :elide-asserts false}}

   :release {:build-options {:ns-aliases {day8.re-frame.tracing day8.re-frame.tracing-stubs}}
             :build-hooks [(teknql.tailwind/compile-release!)]}}
  
  :sw
  {:target :browser
   :output-dir "resources/public/assets/sw-js"
   :asset-path "/assets/sw-js"
   :modules
   {:service-worker
    {:entries [com.mydomain.serviceworker.core]
     :web-worker true}}}}

 :dev-http {5003 "resources/public"}}

thheller18:11:35

since the service worker doesn't support reframe 10x or tailwind anyways that config becomes much simpler

thheller18:11:57

you can then run shadow-cljs watch app sw to run both builds

thheller18:11:06

and have calva connect to the app build as usual

thheller18:11:39

please do make sure that both build use a separate :output-dir though, using the same for both builds will get you in trouble.

andrea18:11:00

That makes sense, I had them separate and couldn't make it work before. I'll try again now, thanks a ton!

andrea18:11:18

By the way, if I could sculpt I'd build you a statue. Is the best way to support you through github or what?

thheller18:11:07

hehe, all the links listed there work yes https://github.com/thheller/shadow-cljs

❤️ 2
andrea11:11:33

Magical, thanks a ton @U05224H0W! @U3BALC2HH FYI the last config shared by Thomas works flawlessly. In order to get a REPL on the :sw I needed an extra

:devtools { :devtools-url "" }
in the :sw module, because the sw is served via https but I'm running the dev env locally. You can switch the REPL in calva via the selector at the bottom and the repl runs in the correct runtime, eg of
(comment (js/console.log "Self is " js/self))
See screenshots.

Lone Ranger19:11:21

holy f*ck 😮

Lone Ranger19:11:53

thank you @U47V0EZF1 @U05224H0W, didn't think it was possible

thheller20:11:20

it is possible with just one build too, calva just doesn't support it. and neither does any other editor to be fair, since its shadow-cljs only feature thats not well documented. 😛

thheller20:11:44

but in case of service workers I'd recommend a secondary build anyways since its unlikely to share much code with your regular build

thheller20:11:47

well ... it can make sense to combine. service workers with caching create sort of a chicken/egg type problem where updating them gets tricky 😛

Lone Ranger21:11:09

Ok so I am a little confused about how to integrate all of this -- you write the service worker code as part of the normal cljs project, do you need a special command to invoke both builds?

thheller21:11:34

what do.you mean by invoke a build? you just build them like normal

Lone Ranger21:11:05

I typically use cider-jack-in-cljs > shadow, then it prompts me for a build

Lone Ranger21:11:32

so do I use it twice, once with each build?

thheller21:11:21

cant say anything about the cider side. no clue how you'd tell it to start both builds

Lone Ranger21:11:03

ok, so "like normal" means on the command line

thheller21:11:29

you could run npx shadow-cljs watch app sw to start both and then cider connect or whatever thats called

Lone Ranger21:11:43

yep perfect, that works

thheller21:11:08

or you start the first with cider and start the second over the repl or the ui

Lone Ranger21:11:26

that would probably work better

Lone Ranger08:11:27

❤️ @U05224H0W you da man. Might have to bump up your corporate sponsorship for this one!! @U47V0EZF1 thanks for making this happen!

Lone Ranger08:11:50

I'm honestly stunned that that even works

thheller08:11:47

beware of service workers though, since they can intercept network requests and reply with cached values. caching is usually not something you want in development, so don't go too crazy with it. 😛

Lone Ranger08:11:22

it's a rich client application and caching is a critical part of it... so better to have a repl than not!

🚀 1
Lone Ranger08:11:59

I had that part entirely in typescript so I could use the webstorm debugger, but the webstorm debugger for service workers really isn't that great

Tanner Hoelzel15:10:51

Hey folks I'm having an issue adding a single dependency to a fresh project. Steps:

$ lein new re-frame re-frame-test
$ cd re-frame-test
$ npm install
$ npm install react-beautiful-dnd
Add [cljsjs/react-beautiful-dnd "12.2.0-2"] to dependencies in shadow-cljs.edn:
{:nrepl {:port 8777}

 

 :source-paths ["src" "test"]

 :dependencies
 [[reagent "1.1.1"]
  [re-frame "1.3.0"]
  [cljsjs/react-beautiful-dnd "12.2.0-2"]

  [binaryage/devtools "1.0.6"]]

 :dev-http
 {8280 "resources/public"
  8290 "target/browser-test"}

 :builds
 {:app
  {:target     :browser
   :output-dir "resources/public/js/compiled"
   :asset-path "/js/compiled"
   :modules
   {:app {:init-fn re-frame-test.core/init}}
   :devtools
   {:preloads []}
   :dev
   {:compiler-options
    {:closure-defines
     { }}}}}}
Refer to dependency in src/re-frame-test/views.cljs:
(ns re-frame-test.views
  (:require
   [re-frame.core :as re-frame]
   [re-frame-test.subs :as subs]
   [cljsjs.react-beautiful-dnd]
   ))

(defn main-panel []
  (let [name (re-frame/subscribe [::subs/name])]
    [:div
     [:h1
      "Hello from " @name]
     ]))
Run:
$ npm run watch
> watch
> npx shadow-cljs watch app browser-test karma-test

shadow-cljs - config: /path/to/parent/re-frame-test/shadow-cljs.edn
No config for build "browser-test" found.
No config for build "karma-test" found.
shadow-cljs - HTTP server available at 
shadow-cljs - HTTP server available at 
shadow-cljs - server version: 2.20.5 running at 
shadow-cljs - nREPL server started on port 8777
shadow-cljs - watching build :app
[:app] Configuring build.
[:app] Compiling ...
[:app] Build failure:
The required namespace "cljsjs.react-beautiful-dnd" is not available, it was required by "re_frame_test/views.cljs".
Compile:
$ npx shadow-cljs compile app
shadow-cljs - config: /path/to/parent/re-frame-test/shadow-cljs.edn
[:app] Compiling ...
The required namespace "cljsjs.react-beautiful-dnd" is not available, it was required by "re_frame_test/views.cljs".
Any ideas? I originally had this issue in an existing project and then discovered that the same issue occurs in a fresh project too. Installing another cljsjs dependency like react-flip-move works. I've ensured there are no other running servers via jps. Also I've noticed that changing the dependency name to react-beautiful-dnd instead of [cljsjs/react-beautiful-dnd] in shadow-cljs.edn makes compilation work, which I do not understand. Thank you

thheller15:10:23

shadow-cljs does not support using cljsjs packages, instead you use the npm packages directly