Fork me on GitHub
#cljs-dev
<
2022-07-02
>
john15:07:50

So I've been running through some of the vanilla scenarios, looking at the main file generated by the :webworker target compiler option. This seems to work in the cljs.main cli repl:

(defn create-worker-body [require-ns]
  (let [pre-loads
        (str "var CLOSURE_UNCOMPILED_DEFINES = {\"cljs.core._STAR_target_STAR_\":\"webworker\"};")]
    (if (empty? goog/basePath)
      (str pre-loads
           "importScripts('" (last (scripts-src)) "');\n")
      (str pre-loads
           "CLOSURE_BASE_PATH = '" goog/basePath "';\n"
           "this.CLOSURE_IMPORT_SCRIPT = (function(global) {\n"
           "    return function(src) {\n"
           "        global['importScripts'](src);\n"
           "        return true;\n"
           "    };\n"
           "})(this);\n"
           "if(typeof goog == 'undefined') importScripts(CLOSURE_BASE_PATH + 'base.js');\n"
           "importScripts(CLOSURE_BASE_PATH + 'deps.js');\n"
           "importScripts(CLOSURE_BASE_PATH + '../cljs_deps.js');\n"
           "goog.require('process.env');\n"
           "goog.require('" require-ns "');\n"))))
But not in a repl started by cljs.repl.browser/repl-env, which calva's deps.edn + Clojurescript built-in for browser config uses: (do (require 'cljs.repl.browser) (cider.piggieback/cljs-repl (cljs.repl.browser/repl-env))) to launch. cljs.repl.browser/repl-env doesn't produce a cljs_deps.js. It produces a brepl_deps.js but when I bring that in (replicating the main fail generated in that scenario) goog.require seems to fail after the imports. And it drops another out/ target folder inside of the existing out/ target folder, for project files - not sure if there's a bug here or not.

john15:07:53

Obviously this just isn't a supported use-case, programmatically spawning web workers with a given build config. It would be a really useful feature though. CLJS could really shine compared to JS for webworker scenarios. Whatever I come up with, I'll try to get it work across as many configurations as possible and maybe it'll be helpful for exploring future support in CLJS.

dnolen20:07:24

there's a bit to digest here - but I don't code splits are relevant here? code splitting is for reducing load time in a complex application

dnolen20:07:44

it doesn't really have any other supported use case

thheller06:07:39

that statement is a bit limiting. you can and (IMHO) should absolutely use :modules for web workers. shadow-cljs has been doing so since the beginning. the basic principle is that you can extract all worker related code into a module with its own entry point. in development and after :advanced you just add one or more importScripts so it automatically imports the common shared code (eg. cljs.core)

thheller06:07:13

the issue with :target :webworker otherwise is that is includes its own duplicated versions of cljs.core and other dependencies. making it unnecessarily large. this might make sense if the worker is truly standalone but often the worker code itself is just tiny and could share most of the other application code

thheller06:07:35

> code splitting is for reducing load time in a complex application

thheller06:07:03

it is essentially fulfilling that purpose here, just also for worker code

john20:07:08

Okay. I was thinking maybe with how the loader system provides a single file to point to for a given artifact, which pulls in the rest of the deps necessary for that artifact, that it might be a smaller thing to wrap a static webworker script around. Or perhaps down the road it might be useful to specify certain modules as being webworker targets, while others as main thread targets, allowing you to have different main entry points for either, so you know to only call document for instance on modules that will be loaded by the main thread, for instance. Looking through the CLJS source, it does appear that the :webworker target option is trying to be smart about modules, but I'm not sure what it's doing. I haven't gotten the combination to work yet.

john20:07:03

Hacking around on it today, it looks like the best flow for figwheel and shadow is to have separate script tags in your html, one for the main thread and one for your webworkers. And when you want to spawn more programmatically, you can basically just do "importScripts('" (last (scripts-src)) "');\n" without all the other stuff and they take care of the rest. So I'm chasing down that same pattern in the default CLJS mode now, seeing if I can get it to work.

john20:07:56

Following these instructions https://github.com/kommen/clojurescript-webworker-demo does seem to work, but it's a bit hamstrung. It doesn't work from a build.edn and it doesn't seem to work if I point the artifact to the out/ folder - only when it's in the base dir.

john21:07:46

I'll try running the build.clj in the out folder itself, or play with :assets-dir and :output-dir and see if that resolves it

john20:07:44

Just brain dumping my explorations here, in case it's helpful - A combination I also got to work was letting calva's default "Clojurescript built-in for Browser" jackin repl, which uses (do (require 'cljs.repl.browser) (cider.piggieback/cljs-repl (cljs.repl.browser/repl-env))) in combination with a build.edn and clj -M -m cljs.main -v -co build.edn -c -s did work, because that resulted in a out/cljs_deps.js that played nice with the static webworker loader.

john21:07:33

But that cljs.repl.browser by itself won't generate a cljs_deps.js, nor can I find a set of params to pass in to the repl-env to force it to create a webworker target that also gens the cljs_deps.js. I'm not sure how to accommodate a webworker target scenario from the cljs.repl.browser path.

john21:07:55

And unfortunately calva does its work by merging in it's own :main-opts, so, at least for calva, I can't figure out how to drive a solution from cljs.main's cli. So I've been trying to get a custom repl sequence defined with the right cljs.repl.browser incantation.

john21:07:13

Anyway, there does seem to be a regression around the :webworker target, https://ask.clojure.org/index.php/10292/console-compiling-clojurescript-target-webworker-release?show=10292#q10292, and I'll try to help chase it down, but in general I just think it'd be awesome if CLJS had some spawn fn with a contract that all the build tools could adhere to as that would make working with webworkers so much easier for users.

john23:07:00

If so, I guess the fix would be to either prevent the conj if :target is :webworker or make the brepl machinery webworker friendly

john23:07:35

Oh wait no, repl? is a static value there

john23:07:43

Maybe that's not the change

john00:07:42

Actually yeah @U050B88UR that was it

john00:07:17

I just tried reverting closure/known-opts to [:target] and that fixed it

john00:07:40

I don't understand why though, as it appears as though :browser-repl is still getting conjed in :thinking_face:

john00:07:35

But I verified it's not... :opts {:target :webworker, :output-to mywebworker.js, :main mywebworker.core} vs :opts {:target :webworker, :output-to mywebworker.js, :main mywebworker.core}

john00:07:09

oooh, :browser-repl is in closure/known-opts, so it ends up in the key selection set regardless of whether repl? is true or not.

john00:07:23

so maybe it should be more like:

(cond-> closure/known-opts
  (not repl?) (disj :browser-repl))

john01:07:35

I can throw up a patch. Do I need a CLJS- issue number?

john04:07:59

So I pushed up a new lib that approaches the UX I'm going for and introduced it on Clojureverse https://clojureverse.org/t/in-mesh-one-step-closer-to-threads-on-the-web/9069. It doesn't have the blocking semantics yet that tau has, but I'm hoping I can rediscover the old sync-xhr via serviceworker hack that I once had working with an early iteration of tau, to get blocking operations back without SABs. Either way though, I think this approaches a UX much more ideal than what we currently have and I'm hoping to reach a place where a user can spawn workers and then ship that out in a lib without having to worry about downstream users having to futz around with the build systems.