This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-12-03
Channels
- # adventofcode (107)
- # announcements (1)
- # asami (14)
- # babashka (67)
- # beginners (89)
- # calva (34)
- # cider (17)
- # clj-kondo (5)
- # cljs-dev (2)
- # clojure (57)
- # clojure-europe (52)
- # clojure-india (1)
- # clojure-italy (1)
- # clojure-losangeles (2)
- # clojure-nl (6)
- # clojure-uk (39)
- # clojurescript (40)
- # community-development (3)
- # conjure (3)
- # cursive (17)
- # datomic (11)
- # docker (13)
- # events (3)
- # figwheel-main (3)
- # fulcro (12)
- # graalvm (7)
- # holy-lambda (7)
- # honeysql (9)
- # introduce-yourself (5)
- # malli (9)
- # minecraft (3)
- # missionary (21)
- # nextjournal (7)
- # off-topic (52)
- # pathom (3)
- # polylith (11)
- # portal (3)
- # re-frame (21)
- # reagent (34)
- # reclojure (7)
- # reitit (1)
- # reveal (11)
- # shadow-cljs (68)
- # tools-build (12)
- # tools-deps (5)
- # vim (4)
- # xtdb (9)
Hi! I was doing some setup for local development to minimize the chances of accidentally releasing a build with wrong configuration and encountered a strange behaviour. I've tried to override :closure-defines
from my build hook with values it reads from local file, but updates were not propagating into the build. Here are the snippets:
app/dev.clj
(defn load-dev-vars
{:shadow.build/stage :configure}
[{:keys [:shadow.build/mode] :as build-state} relative-path]
(let [conf-path (File. (:project-dir build-state) relative-path)]
(if (and (= :dev mode) (.exists conf-path))
(let [overrides (edn/read-string (slurp conf-path))]
(println "Applying vars override:" overrides)
(update-in build-state [:shadow.build/config :closure-defines]
merge overrides))
build-state)))
shadow-cljs.edn
{...
:builds
{:front
{:target :browser
:modules {:main {:init-fn app.system/go}}
;; Dev vs Prod build vars. Override via extra config.
:build-hooks [(app.dev/load-dev-vars "local-defines.edn")]
:closure-defines {app.config/api-root "/api"}}}}
Inspecting build-state showed that overrides were applied as expected, but build contained original values... Looking at the shadow-cljs source I found that at :configure
stage defines are injected into other part of build-state
- :complier-options
. Changing the path from [:shadow.build/config :closure-defines]
to [:compiler-options :closure-defines]
indeed fixed the issue.
Why not just add my defines into :dev :closure-defines
in shadow-cljs.edn - I don't want to commit local configuration into version control and/or stash the changes on every pull. With an additional file, added into git and docker ignores, it's much easier to manage.
My question: is it expected behavior or just code ordering issue in configure stage of shadow-cljs? Maybe it was not supposed that users would change build configuration in hooks?..at the point the :configure
hook runs the build config is already applied and as such modifying it has no effect
Hello. Yeah, thanks, I've figured that out already 🙂
Just wondered if it's a bug or I was not supposed to change build config in hooks.
Hello I'm getting this error when I try to call postMessage on a web worker
subs.cljs:94 Uncaught ReferenceError: document is not defined
at subs.cljs:94
at worker.js:1457
at Array.forEach (<anonymous>)
at Object.env.load (worker.js:1453)
at worker.js:1489
(anonymous) @ subs.cljs:94
(anonymous) @ worker.js:1457
env.load @ worker.js:1453
(anonymous) @ worker.js:1489
the webworker itself looks as follows
(ns quagga.components.dataview.worker)
(defn init []
(set!
js/self.onmessage
(fn [^js e]
(js/console.log "message received" (.. e -data)))))
I've added it to the modules like this
:worker
{:init-fn quagga.components.dataview.worker/init
:depends-on #{:shared}
:web-worker true}
I'm calling postMessage
as follows on the repl
(let [worker (js/Worker. "/js/quagga/worker.js")] (.. worker (postMessage "hello world")))
What comprises the module :shared
? Given the stacktrace, there's some subs.cljs
file in there that tries to reference js/document
, which does not exist in web workers.
I tried to make the web worker module depend on :depends-on #{}
but then it says
ExceptionInfo: two modules without deps, please specify which one is the default
Oh, interesting. Maybe I misunderstand something then.
Shadow-cljs docs say:
> You can leave the `:entries` of the `:shared` module empty to let the compiler figure out which namespaces are shared between the other modules.
But that subs.cljs
still comes from somewhere even though your web worker ns clearly doesn't depend on it. @U05224H0W Could it be a bug?
are there other modules that may cause stuff to be moved into :shared
that the worker doesn't like?
:devtools
{:preloads [day8.re-frame-10x.preload],
:watch-dir "resources/public/css/"},
:modules
{:home
{:depends-on #{:shared}, :entries [quagga.components.home.init]},
:organization
{:depends-on #{:shared},
:entries [quagga.components.organization.init]},
:settings
{:depends-on #{:shared},
:entries [quagga.components.organization.settings.init]},
:shared {:entries []},
:tabbed-dataview
{:depends-on #{:shared},
:entries [quagga.components.dataview.init]},
:user-management
{:depends-on #{:shared},
:entries [quagga.components.organization.users.init]}
:worker
{:init-fn quagga.components.dataview.worker/init
:depends-on #{:shared}
:web-worker true}},
:output-dir "resources/public/js/quagga/",
yeah so so since you only have :shared
anything that two of the other modules may use will get moved to :shared
that the :shared
and :worker
depend on but have all the others only depend on :shared
Ah, I see. At least to me, the wording in the documentation made it seem like :shared
has namespaces that are shared by all other modules that depend on :shared
.
Thanks!
yeah a namespace can only ever be in one module so if say something from :home
and :settings
share a required namespace the only place it can go is :shared
.
> a namespace can only ever be in one module Now that you said it out loud, it became obvious to me as well. :)
unfortunately, I'm still getting the same error with this setup
:modules
{:home
{:depends-on #{:shared}, :entries [quagga.components.home.init]},
:organization
{:depends-on #{:shared},
:entries [quagga.components.organization.init]},
:settings
{:depends-on #{:shared},
:entries [quagga.components.organization.settings.init]},
:shared {:entries []
:depends-on #{:base}},
:base {:entries []}
:tabbed-dataview
{:depends-on #{:shared},
:entries [quagga.components.dataview.init]},
:user-management
{:depends-on #{:shared},
:entries [quagga.components.organization.users.init]}
:worker
{:init-fn quagga.components.dataview.worker/init
:depends-on #{:base}
:web-worker true}}
are you sure none of the namespaces used by the quagga.components.dataview.worker
ns end up including the problem ns?
there's a line in worker.js
that looks like this
SHADOW_ENV.load({}, ["goog.debug.error.js",..."day8.re_frame_10x.inlined_deps.re_frame.v1v1v2.re_frame.subs.js","shadow.cljs.devtools.client.browser.js","shadow.module.base.append.js","quagga.components.dataview.worker.js","shadow.module.worker.append.js"]);
(I have eliminated some code) could this mean that re-frame-10x might be adding some undesired js to the worker.js somehow?so, I commented out :devtools
from my app
build
;; :devtools
;; {:preloads [day8.re-frame-10x.preload],
;; :watch-dir "resources/public/css/"},
ah its is the preloads yeah. you can move the preloads to the :shared
module. just :shared {:entries [] :depends-on #{:base} :preloads [day8.re-frame-10x.preload]}
At work, we noticed that bumping Shadow to 2.16.6
resulted in a 50% increase in size of our advanced build.
The increase seem to have been introduced with 2.15.13
which uses a newer version of the CLJS compiler with some various Closure updates.
Did anyone notice an increase in size as well?
I'm trying to build them but it fails like this:
Hook [0 shadow.cljs.build-report/hook] failed in stage :flush
NoSuchFileException: .shadow-cljs/builds/psap/release/out/main.js
Does that ring any bell?(also hi to my original hometown and thanks for making shadow-cljs 👋😄)
ah lemme try the standalone build
yup, it's :node-library
ah, too bad 😞
Indeed, the standalone run failed, as well
I think I was able to convert it into a :browser
💪 Still need to confirm whether the regression is reproducible this way, too. Stay tuned.
try setting :js-options {:js-provider :require}
. it won't actually run in the browser that way but the build report won't suddenly contain all the bundled npm depenendeices
OK nice, the size increase still reproduces that way
And we got some build reports now
Looks like the main culprit is the application code itself
Its build output pretty much doubled in size
Maybe more inlining going on or something like that? :thinking_face: Any ideas how to dig deeper?
Maybe of note that said code is all .cljc
Ah but I can also see similar increases for the .cljs
portions
Alright, then it's probably related to something unusual we're doing 😉 I wasn't asking about guesses moreso than hints how to investigate this further. The build report is a great help already, thanks a lot!
I guess we'll revert the update for now and try to get to the bottom of this. If we do or are able to come up with an isolated repro, we'll get back to you!
the only thing I can think of is running the build with --pseudo-names
and looking at the output trying to figure out where it increase is coming from
--pseudo-names
will make it huge regardless but at least you can figure out what the code was before optimizations
Great, thanks!