Fork me on GitHub
#shadow-cljs
<
2023-10-13
>
bru07:10:06

potential bug(s), couldn't find any literature on the interwebs in general and on github in particular: I have a relatively large codebase that's been evolving on shadow-cljs for years; it's targeting the browser and has grown to include multiple modules (in the shape of web workers), node module dependencies and some wasm for added fun. I've been running on shadow-cljs 2.25.2 for a while and was recently trying to catch up with the latest development but two things happen: 1. if try to move to shadow-cljs 2.25.3 I get the following error: > aborted par-compile, [:shadow.build.classpath/resource "app/events.cljs"] still waiting for #{app.events.icloud app.fx.dom app.fx.icloud} > {:aborted [:shadow.build.classpath/resource "app/events.cljs"], :pending #{app.events.icloud app.fx.dom app.fx.icloud}} > ExceptionInfo: aborted par-compile, [:shadow.build.classpath/resource "app/events.cljs"] still waiting for #{app.events.icloud app.fx.dom app.fx.icloud} I know that's usually related to the build taking too long, but my eyes can't find anything in the 2.25.3 https://github.com/thheller/shadow-cljs/compare/bb586a0fab475c13c6d090164c4122f2782308e3...a6b22dd943506bc5b208326adb2b5d7e15329e6c to justify that error Also, this error goes away if I remove the worker module and keep the app monolithic 2. regardless if I take the modular or monolithic approach, upgrading to 2.25.8 gives me this error: > Closure compilation failed with 1 errors > --- node_modules/my_cool_webassembly_component/wasm.js:49 > Illegal redeclared variable: module here I see that 2.25.8 is mainly concerned with node targets, which is not my case... so I'm a bit puzzled. Any pointers would be appreciated 🙂

bru07:10:26

hm... and now I can't get rid of the wasm.js error even if I rollback to 2.25.2. This is weird

bru08:10:10

the plot thickens: cleaning up .shadow-cljs doesn't help (I know, I know, it should never be done anyway)... I'm still stuck with the error in 2.25.2... but, if I revert to 2.24.1 the error goes away, and I can then upgrade to 2.25.2 again (and if I stay monolithic go all the way up to 2.25.7). Now the question is, does shadow-cljs (or the closure compiler underneath) maintain any form of state between compilations that could justify this behaviour?

thheller09:10:56

all state/cache is kept in .shadow-cljs/builds, so if you wiped that and restarted shadow-cljs all state is gone. changing shadow-cljs versions also automatically invalidates all the caches

thheller09:10:14

my main suspect is always not actually shadow-cljs but npm

thheller09:10:44

do you keep a proper package-lock.json file, or is that gitignored or deleted?

thheller09:10:01

without a lockfile it is pretty much random which sort of npm package setup you get

thheller09:10:50

nothing related to :par-compile has changed in many years, so I'm not sure what would cause that

thheller09:10:24

about the .wasm error, what do you do in line 49 in that file 😛

thheller09:10:58

the :par-compile thing was reported a few days ago, so I'm curious https://clojurians.slack.com/archives/C6N245JGG/p1696419053913689

thheller09:10:36

I don't have any guess though, but willing to help you diagnose. normally the exception thrown should give enough info to tell whats going on

thheller10:10:04

well I have one guess .. that is macros. if you use any kind of weird macros, those that keep state outside the compilation in a atom somewhere, then shadow-cljs doesn't know about it and all cache guarantees go out the window

bru11:10:02

hm, interesting. I don't have many macros, but I do have one that I use everywhere to log, there shouldn't be much state there but let me check

thheller11:10:08

still waiting for #{app.events.icloud app.fx.dom app.fx.icloud} means that compilation of those namespaces is preventing the others from getting compiled

bru11:10:24

the wasm.js error is very odd: I actually don't have anything in line 49 of that file, I thought that maybe it's looking at a sourcemap. Anyway, one problem at a time, I'll go check those macros

thheller11:10:28

so check those namespaces first 😛

bru11:10:52

will do :saluting_face:

thheller11:10:59

line numbers may be off by 1 or 2, so check those too

bru11:10:01

yeah, that file is all assignments, the closest to that line all look like:

NodeType[NodeType["ICE_TABLE_NODE"] = 40] = "ICE_TABLE_NODE";
    NodeType[NodeType["ICE_TABLE_PIPE_MARKER_NODE"] = 41] = "ICE_TABLE_PIPE_MARKER_NODE";
    NodeType[NodeType["ICE_TABLE_DELIMITER_MARKER_NODE"] = 42] = "ICE_TABLE_DELIMITER_MARKER_NODE";

bru11:10:34

but I'll focus on those macros now

bru11:10:22

@U05224H0W right... found the issue with par-compile: I have no idea how it got there, but I had one namespace :require itself 😊 so that's solved now

bru11:10:10

(the amazing thing is how it manage to work before up till 2.25.2! 😄 )

thheller11:10:10

hmm that still shouldn't be legal.

thheller11:10:47

as in I mean that should throw an error

bru11:10:26

yes I understand. Maybe it doesn't because it happens in the web worker module instead of the main one?

thheller11:10:06

hmm nope, just accepted as is. I will add a check for that

bru11:10:31

this is the relevant part of shadow-cljs.edn, in case it can help you contextualise:

:builds
 {:app
  {:target :browser

   :modules
   {:shared
    {:entries []}

    :app
    {:init-fn app.core/init
     :preloads [day8.re-frame-10x.preload]
     :depends-on #{:shared}}

    :docker
    {:init-fn app.docker/init
     :depends-on #{:shared}
     :web-worker true}}

   ;; :module-hash-names 8
   :build-hooks
   [(shadow.hooks/index)
    (shadow.hooks/svg)
    (shadow.hooks/robots)]

   :compiler-options
   {:output-feature-set :es6
    :optimizations :advanced
    :infer-externs :auto}

   :js-options {:js-provider :shadow}

   :devtools {:loader-mode :eval}

thheller11:10:53

doens't matter at all. just a namespace requiring itself blows up in any context

bru11:10:07

🤷 😄

bru11:10:51

now I can joyfully turn to the wasm.js issue. Fun!

thheller11:10:07

make sure you are looking at the actual node_modules/my_cool_webassembly_component/wasm.js, not the file that may produce this

bru11:10:46

yeah that's what I though too. I'm using yarn and it does have a habit of keeping a cache of the modules...

bru11:10:28

thanks for all the precious advice!

Stef Coetzee07:10:35

👋 I want to serve two builds simultaneously (on separate ports): one for the CLJS app and one for Portfolio. Can this be done and if indeed how would one approach it?

1
thheller08:10:05

by serve you mean http server?

👍 1
thheller08:10:42

https://shadow-cljs.github.io/docs/UsersGuide.html#dev-http :dev-http takes a map, so you just add a second map entry

thheller08:10:55

:dev-http {8000 "public" 8001 "foo"}

Stef Coetzee08:10:55

In the watcher command, both builds are specified:

npx shadow-cljs -d cider/cider-nrepl:0.28.5 watch :app :portfolio
In the :dev-http map, two ports are specified:
{:source-paths ["src" "portfolio"]

 :dependencies [[binaryage/devtools "1.0.7"]
                [reagent "1.2.0"]
                [no.cjohansen/portfolio "2023.07.15"]
                [fork "2.4.3"]]

 :dev-http     {9800 ["public" "classpath:public"]
                8080 "public"}

 :builds       {:app
                {:target     :browser
                 :output-dir "public/js"
                 :asset-path "/js"
                 :modules    {:main {:init-fn forked.core/init}}}

                :portfolio
                {:target  :browser
                 :output-dir "public/js"
                 :asset-path "/js"
                 :modules {:main {:init-fn portfolio/init}}}}}
The 9800 roots are configured per https://github.com/cjohansen/portfolio#shadow-cljs for working with Shadow CLJS. The part I'm stuck on: how can one tell Shadow CLJS which build to serve on a given port?

thheller08:10:21

shadow-cljs doesn't care what you serve where. it all depends on where you put your JS files

thheller08:10:40

in general you should not use the same :output-dir for two builds

1
Stef Coetzee08:10:57

Gotcha, that's where I went astray.

thheller08:10:02

say you put :output-dir "public/portfolio" in one config

thheller08:10:19

then you only need one server and access the /portfolio/main.js in the one html and /js/main.js in the other

thheller08:10:59

also need to adjust :asset-path "/portfolio" accordingly

Stef Coetzee10:10:08

Thanks @U05224H0W! I've set it up as follows, which yields the desired result:

{...

 :dev-http     {9800 {:roots            ["public" "classpath:public"]
                      :push-state/index "portfolio.html"}

                8080 "public"}

 :builds       {:app
                {:target     :browser
                 :output-dir "public/js"
                 :asset-path "/js"
                 :modules    {:main {:init-fn forked.core/init}}}

                :portfolio
                {:target     :browser
                 :output-dir "public/portfolio"
                 :asset-path "/portfolio"
                 :modules    {:main {:init-fn portfolio/init}}}}}

thheller10:10:09

I'd personally just use a different folder altogether, but that is up to you.

{...

 :dev-http {9800 ["portfolio" "classpath:public"]
            8080 "public"}

 :builds {:app
          {:target :browser
           :output-dir "public/js"
           :modules {:main {:init-fn forked.core/init}}}

          :portfolio
          {:target :browser
           :output-dir "portfolio/js"
           :modules {:main {:init-fn portfolio/init}}}}}

thheller10:10:31

so everything portfolio related goes into the portfolio dir, the rest is in public