When I run npx shadow-cljs watch stories I get the following error:

npx: installed 91 in 3.066s
shadow-cljs - config: /Users/.../shadow-cljs.edn  cli version: 2.8.52  node: v10.16.3
shadow-cljs - connected to server
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:436).
shadow.cljs.devtools.cli

Looking at this...

https://github.com/thheller/shadow-cljs/issues/180

I am running [org.clojure/core.async "0.3.465"] so it should still work...

@studio not related to that at all

I'm guessing you are running an embedded server that you started via (shadow.cljs.devtools.server/start!)?

if so you must also (:require [shadow.cljs.devtools.cli]) in the ns that starts the server to ensure that it is loaded

and FWIW shadow-cljs uses [org.clojure/core.async "0.4.500"]

Ah, yes that seems to have fixed the issue

[shadow.cljs.devtools.cli]
[shadow.cljs.devtools.api :as shadow-cljs]
[shadow.cljs.devtools.server :as shadow-server]

Then get No config for build "stories" found. Build id required. - It must not be picking up my stories config in shadow-cljs.edn

well is it defined correctly?

{:lein true
 :builds {:frontend {:target :browser
                     :output-dir "resources/app/public/js"
                     :asset-path "/js"
                     :modules {:main {:entries [frontend.core]}}}}
          :stories  {:target :npm-module
                     :output-dir "resources/app/public/js"
                     ;; storybook already handles all reloading
                     :devtools {:enabled false}}}}

helps if you use paredit or so. parens don't match

properly indented you get

{:lein true
 :builds {:frontend {:target :browser
                     :output-dir "resources/app/public/js"
                     :asset-path "/js"
                     :modules {:main {:entries [frontend.core]}}}}
 :stories {:target :npm-module
           :output-dir "resources/app/public/js"
           ;; storybook already handles all reloading
           :devtools {:enabled false}}} }

Ah, sorry, still getting the hang of the syntax. There's an Atom Paredit so will look into that

parinfer is also good and easier to get started with

saves you from issues like that one

Storybook have updated their story syntax, once I get this up and running I could update your example code?

if there aren't too many changes sure 🙂

They have cleaned it up, looks a lot easier to create stories...

https://storybook.js.org/docs/basics/writing-stories/

Hmm still get No config for build "stories" found. :face_with_monocle:

make sure you have your parens correctly balanced

in the above :stories wasn't in :builds, I didn't paste the corrected version

just a version that highlighted your mistake better

Right, parinfer has saved the day

hey @thheller, some of my devs are seeing the "too many open files" error during shadow builds... they have large ulimit -n ( 30720, which seems like it should be enough). blatting their .shadow-cljs directory seems to fix the problem temporarily, but it still seems to crop up after a while. were on cli version: 2.8.52 node: v12.5.0 (which is later than https://github.com/thheller/shadow-cljs/issues/261 fix)... any ideas ?

if I could prevent people from ever deleting .shadow-cljs I would ...

NEVER do that please

makes it impossible to debug what is happening

next time you run into that at least make a copy or so

it is likely creating too many cache files? which means you are accessing too many npm files maybe?

we can make a copy next time it happens...

it creates 2 cache files per npm file (one source, one source map)

so if you access 1000 npm files thats already 3k

and not unlikely at all to access more than 1k

but yeah .shadow-cljs is just cache, which will just be written again so it should end up at the same values after one build

do you maybe run several builds in parallel or so?

find .shadow-cljs | wc is returning 7118 for me atm... well within my 32k ulimit

last time I checked there were no file leaks anymore. so in case it happens again try to find out which files are open

thats only the cache files though. it'll still have the files from node_modules open

use lsof or something maybe?

anyway, i've given instructions to keep .shadow-cljs next time it happens and get an lsof dump

we don't run several builds in parallel - we do have a bunch of different builds and we do feed --config-merge EDN to those builds

here's an lsof dump @thheller - almost all the files are .jars from ~/.m2

there aren't any source files from the project directory, or cache files from the .shadow-cljs directory there

hmm odd. how many jars are there?

10440

what the heck? the same open multiple times I guess?

yep - there are definite repeats

every jarfile i've looked at so far is open twice

oh, wow - promesa-1.9.0.jar is open 142 times!

and reagent-0.8.1.jar 362 times

weird

is it shadow opening jarfiles, or is that down to the cljs compiler ?

well it is opening whenever a file in them is accessed yes

but the classloader is doing that

I would expect it to close right after?

hmm... well something is going awry - does each run of the compiler create a new classloader perhaps ?

i can't remember whether random access in a zipfile makes any sense and thus whether it would make sense for a classloader to hold jars open to load future class requirements from

presumably this is an issue of shadow server mode though, and restarting the shadow server should make everything good

we'll try that next time it happens

Hey @thheller! Not quite sure how to phrase this question/concern but I’ve been having quite a few very cryptic and laborious-to-isolate errors and I wonder whether shadow-cljs could report them in a more user-friendly or easy-to-debug way. I’ve been able to work through them with a mix of 1) (set! (.-onerror js/document) (fn [...] (log-pretty-error ...))), 2) bisecting the code by commenting out parts till I get to the error, and 3) just restarting the build. 2) and 3) have proven particularly time-consuming. Here’s an example of something (I still don’t know what it is — have to do good ol’ option 2 and 3 till I figure out) :

[:local.client.search] Compiling ...
[:local.client.search] Build failure:
ClassCastException: java.lang.String cannot be cast to clojure.lang.Named

        clojure.core/namespace (core.clj:1597)
        clojure.core/namespace (core.clj:1597)
        shadow.build.compiler/write-cached-cljs-resource/spec-filter-fn--11841 (compiler.clj:707)
        shadow.build.compiler/write-cached-cljs-resource/fn--11844 (compiler.clj:713)
        clojure.core/fn--8429/fn--8431 (core.clj:6840)
        clojure.core.protocols/iter-reduce (protocols.clj:49)
        clojure.core.protocols/fn--8140 (protocols.clj:75)
        clojure.core.protocols/fn--8140 (protocols.clj:75)
        clojure.core.protocols/fn--8088/G--8083--8101 (protocols.clj:13)
        clojure.core/reduce (core.clj:6828)
        clojure.core/fn--8429 (core.clj:6830)
        clojure.core/fn--8429 (core.clj:6830)
        clojure.core.protocols/fn--8167/G--8162--8176 (protocols.clj:175)
        clojure.core/reduce-kv (core.clj:6856)
        clojure.core/reduce-kv (core.clj:6847)
        shadow.build.compiler/write-cached-cljs-resource (compiler.clj:711)
        shadow.build.compiler/write-cached-cljs-resource (compiler.clj:677)
        shadow.build.compiler/maybe-compile-cljs (compiler.clj:832)
        shadow.build.compiler/maybe-compile-cljs (compiler.clj:762)
        shadow.build.compiler/par-compile-one (compiler.clj:895)
        shadow.build.compiler/par-compile-one (compiler.clj:850)
        shadow.build.compiler/par-compile-cljs-sources/fn--11897/iter--11919--11923/fn--11924/fn--11925/fn--11926 (compiler.clj:968)
        clojure.core/apply (core.clj:665)
        clojure.core/with-bindings* (core.clj:1973)
        clojure.core/with-bindings* (core.clj:1973)
        clojure.core/apply (core.clj:669)
        clojure.core/bound-fn*/fn--5749 (core.clj:2003)
        java.util.concurrent.FutureTask.run (FutureTask.java:266)
        java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
        java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
        java.lang.Thread.run (Thread.java:748)

[:local.client.search] Compiling ...
[:local.client.search] Build failure:
ClassCastException:

[:local.client.search] Compiling ...
[:local.client.search] Build failure:
ClassCastException:

the plot thickens. after restarting there are only 546 open jarfiles, but the build still fails with too-many-open-files - so it looks like the open jarfiles are not the root cause of the problem. whatever is the cause is not yet clear - there are only 2044 files in the .shadow-cljs directory

currently trying to capture an lsof dump during compilation, since perhaps whatever open files are causing the issue are getting closed after compilation fails

@alexandergunnarson that error indeed sucks but I have never seen that so kinda not sure whats happening

it appears to be spec related maybe?

that part of the code deals with caching specs. do you do any black magic with spec? custom implementations or so?

Interesting; yeah now I’m seeing the spec-filter-fn

I do a few black magics with it 🙂

I think this was a case of for whatever reason I was defining a spec incorrectly (e.g. transposing docstring with keyword name) and it got swapped into the spec cache

So I have a slightly custom s/def that does like (s/def :whatever/spec-name "docstring" {:meta-stuff "if you want"} some-spec)

I was erroneously doing (s/def "docstring" :whatever/spec-name)

And I think that bad data got swapped in

hmm weird that that is allowed

but yes that would cause that error

Makes sense! Well anyway I will keep you posted of similarly odd/cryptic errors I get haha

Maybe there’s a pattern in terms of usability enhancements

well if you do stuff that no one else has done before there likely aren't good error messages there 😉

Heh good point 😉

https://github.com/thheller/shadow-cljs/commit/356289b3ae44753099d9cbe5f79765af5cdf69e2

won't break the entire build if it happens again 😉

Okay cool haha — thank you thank you! If you were a customer support person I’d rate you a 12/10!

Too bad there’s not much money in open source… even if it is an awesome tool… ah well, Patreon

+1 on that: thheller provides tons of great support here. https://www.patreon.com/thheller for those who want to show a little love