Fork me on GitHub
#shadow-cljs
<
2023-11-25
>
ns17:11:38

Hi, I'm wondering if someone can help me setup a web worker with shadow cljs. The way my modules are setup is I have a main "app" module (which acts as shared) and then other modules which depend on it. When I try to add web worker module usually I end up with errors such as "document is not defined" and "history is not defined". I think the problem is that things that are supposed to be in "app" module only end up in worker module as well (for example pez/clerk package which references history) Shadow-cljs web worker user guide (https://shadow-cljs.github.io/docs/UsersGuide.html#_web_workers) has "shared" module which all others depend on including the web worker but if I try to add one I end up with "two modules without deps, please specify which one is the default". Attached is my shadow cljs config, any help would be very much appreciated:

{:nrepl {:port 8777}

 :source-paths ["src" "test"]

 :dependencies
 [[reagent "1.1.0"]
  [re-frame "1.2.0"]
  [day8.re-frame/tracing "0.6.2"]
  [re-com "2.13.2"]
  [bidi "2.1.6"]
  [clj-commons/pushy "0.3.10"]
  [re-pressed "0.3.1"]
  [breaking-point "0.1.2"]
  [binaryage/devtools "1.0.3"]
  [day8.re-frame/re-frame-10x "1.1.11"]
  [day8.re-frame/http-fx "0.2.3"]
  [com.cemerick/url "0.1.1"]
  [org.clojars.mmb90/cljs-cache "0.1.4"]
  [day8.re-frame/async-flow-fx "0.3.0"]
  [herb "0.10.1"]
  [pez/clerk "1.0.0"]
]

 :dev-http
 {8280 "resources/public"
  8290 "target/browser-test"}

 :builds
 {:app
  {:target     :browser
   :output-dir "resources/public/js/compiled"
   :module-hash-names true
   :module-loader true
   :modules

     {:app {:init-fn admin.core/init}

      :posts
      {:entries [admin.posts.views]
       :depends-on #{:app}}

      :analytics
      {:entries [admin.analytics.views]
       :depends-on #{:app}}

      :sessions
      {:entries [admin.sessions.views]
       :depends-on #{:app}}}



   :devtools
   {:preloads [day8.re-frame-10x.preload]}
   :dev
   {:asset-path "/js/compiled"
    :compiler-options
    {:closure-defines
     {re-frame.trace.trace-enabled? true
      day8.re-frame.tracing.trace-enabled? true
      re-com.config/root-url-for-compiler-output ""}}}
   :release
   {:asset-path "/assets/js/compiled"
    :build-options
    {:ns-aliases
     {day8.re-frame.tracing day8.re-frame.tracing-stubs}}}}}}

thheller18:11:32

it might be the preloads causing that

thheller18:11:54

or your require structure makes it so that code the worker uses also references dom stuff

thheller18:11:54

you probably want something like this

{:shared
 {:entries []}

 :app
 {:init-fn admin.core/init
  :preloads [day8.re-frame-10x.preload]
  :depends-on #{:shared}}

 :posts
 {:entries [admin.posts.views]
  :depends-on #{:app :shared}}

 :analytics
 {:entries [admin.analytics.views]
  :depends-on #{:app :shared}}

 :sessions
 {:entries [admin.sessions.views]
  :depends-on #{:app :shared}}

 :worker
 {:init-fn admin.worker/init
  :web-worker true
  :depends-on #{:shared}}}

thheller18:11:00

with the :devtools entry removed entirely

thheller18:11:11

basically you just need to make sure that nothing from the worker references anything from the other modules (except shared) directly, so no (:require [admin.core :as core]) or whatever

ns18:11:10

@U05224H0W Hi, thank you so much for replying! I've tried that and now I get the following error:

ns18:11:29

app.js:1 Uncaught ReferenceError: SHADOW_ENV is not defined
    at app.js:1:1

ns18:11:56

which is this line:

SHADOW_ENV.evalLoad("shadow.module.app.prepend.js", false, "\nshadow.loader.set_load_start(\x27app\x27);");

ns18:11:09

but the worker now looks much better. worker.js:

SHADOW_ENV.evalLoad("shadow.module.worker.prepend.js", false , "\nshadow.loader.set_load_start(\x27worker\x27);");
SHADOW_ENV.evalLoad("admin.worker.js", true , "goog.provide(\x27admin.worker\x27);\nadmin.worker.init \x3d (function admin$worker$init(){\nreturn self.addEventListener(\x22message\x22,(function (e){\nreturn postMessage(e.data);\n}));\n});\n");
SHADOW_ENV.evalLoad("shadow.module.worker.append.js", false , "\nshadow.cljs.devtools.client.env.module_loaded(\x27worker\x27);\n\nshadow.loader.set_loaded();\ntry { admin.worker.init(); } catch (e) { console.error(\x22An error occurred when calling (admin.worker/init)\x22); throw(e); };\nSHADOW_ENV.setLoaded(\x22shadow.module.worker.prepend.js\x22);\nSHADOW_ENV.setLoaded(\x22admin.worker.js\x22);\nSHADOW_ENV.setLoaded(\x22shadow.module.worker.append.js\x22);");

thheller18:11:15

in the HTML you need to load the shared.js file prior to the app.js file

thheller18:11:34

the worker does that itself but the app.js file cannot as of now

ns19:11:44

I added shared.js to HTML and that took care of that error but now when I try to create the worker:

(let [worker (js/Worker. "/js/compiled/worker.js")]
                         (.. worker (addEventListener "message" (fn [e] (js/console.log e))))
                         (.. worker (postMessage "hello world")))
I get the following error:
worker.js:1 Uncaught ReferenceError: SHADOW_ENV is not defined
    at worker.js:1:1
which is this line:
SHADOW_ENV.evalLoad("shadow.module.worker.prepend.js", false, "\nshadow.loader.set_load_start(\x27worker\x27);");

thheller20:11:30

oh oops. i forgot the :web-worker true in the build config

thheller20:11:47

for the worker module

ns20:11:38

The worker says hello world 🙂 . Thank you so much for all your help!

👍 1
ns18:11:18

@U05224H0W there is still one problem unfortunately

ns18:11:58

when I do release I use :module-hashnames option

ns18:11:47

however worker tries to load importScripts("shared.js") without hash, even though I load the hashed version in html script tag

ns18:11:58

<script>
    var webWorkerFile="/assets/js/compiled/worker.84D0DF83F3955C46E01ED037C6E5F7C6.js";
</script>
<script src="/assets/js/compiled/shared.7D87480495688DE84B39E87B1A2BDD8F.js"></script>
<script src="/assets/js/compiled/app.08E387C98C9872E160C80225449C162E.js"></script>

ns18:11:17

and therefore I get the following error:

Uncaught DOMException: Failed to execute 'importScripts' on 'WorkerGlobalScope': The script at '/assets/js/compiled/shared.js' failed to load.
    at /assets/js/compiled/worker.84D0DF83F3955C46E01ED037C6E5F7C6.js:1:1

ns18:11:11

I did see the option to use

shadow-cljs release app --config-merge '{:release-version "v1"}'
instead of hashed names, but I'm wondering, since shared module is already included via scripts, can the importScripts("shared.js") be turned off in worker script and would that solve the issue? Probably not, but just checking

ns18:11:19

Or can the worker be somehow made to load shared.hash.js (if it cannot do without it)

thheller20:11:54

hashes are tricky as adding the hash to the file changes the hash of the file 😛

thheller20:11:02

it cannot do without it, so it needs to be imported

thheller20:11:47

if you want to setup a repro I can take a look, I just might have forgotten to consider the worker case

thheller20:11:05

nvm I can reproduce it

ns21:11:27

Thank you for looking into it, I got hungry baby twins screaming at me so no time to setup a repro 🙂