This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-08-20
Channels
- # announcements (3)
- # beginners (63)
- # calva (1)
- # cider (24)
- # clj-kondo (98)
- # cljdoc (8)
- # cljsrn (19)
- # clojure (106)
- # clojure-conj (2)
- # clojure-europe (5)
- # clojure-italy (5)
- # clojure-nl (8)
- # clojure-spec (8)
- # clojure-uk (13)
- # clojuredesign-podcast (7)
- # clojurescript (54)
- # core-async (1)
- # cursive (3)
- # data-science (1)
- # datomic (19)
- # fulcro (7)
- # hoplon (1)
- # off-topic (3)
- # re-frame (13)
- # reitit (1)
- # shadow-cljs (234)
- # test-check (10)
- # tools-deps (59)
- # unrepl (1)
- # yada (20)
I have an inconsistently occurring bug to report --
In Emacs, if I start M-x cider-jack-in-cljs
it usually (90% of time) fails with below output. One in 10 runs inexplicably succeeds.
[:app] Configuring build.
[:app] Compiling ...
[:app] Build completed. (608 files, 1 compiled, 0 warnings, 3.68s)
To quit, type: :cljs/quit
user> [:selected :app]
shutting down ...
cljs.user>
*** Closed on Mon Aug 19 18:38:22 2019 ***
If I delete the .shadow-cljs
folder before running cider-jack-in-cljs
, it always succeeds.
[:app] Build completed. (608 files, 607 compiled, 0 warnings, 16.99s)
To quit, type: :cljs/quit
user> [:selected :app]
cljs.user>
Starting via lein dev
works fine 100% of the time.
Deleting the folder .shadow-cljs/builds/app/dev/ana/northstar
seems to raise the success rate to around ~50%output from the nrepl-server buffer:
shadow-cljs - server pid exists but server appears to be dead, proceeding without server.
shadow-cljs - config: /Users/quest/repos/ns/browser/shadow-cljs.edn cli version: 2.8.46 node: v12.8.0
shadow-cljs - running: lein update-in :dependencies conj [nrepl "0.6.0"] -- update-in :dependencies conj [cider/piggieback "0.4.0"] -- update-in :dependencies conj [refactor-nrepl "2.4.0"] -- update-in :dependencies conj [cider/cider-nrepl "0.21.1"] -- run -m shadow.cljs.devtools.cli --npm -d nrepl:0.6.0 -d cider/piggieback:0.4.0 -d refactor-nrepl:2.4.0 -d cider/cider-nrepl:0.21.1 server
shutting down..
* Whether I launch the browser or not (when prompted) seems to have no effect.
* Deleting .shadow-cljs/builds/app/dev/ana/cljs
seems to raise success rate to 100%. I notice that it's a 5MB+ folder, which probably causes the compilation step to take longer.
* However deleting .shadow-cljs/builds/app/dev/ana/re_frame
still results in a noticeable amount of failure -- but it's only a 500K library, so maybe similar to deleting my ana/northstar/browser
(project's) folder
My guess is some sort of race condition that's negated by sufficiently increased compilation time. Unfortunately the guess is all I have, but I'll assist with troubleshooting however possible
@quest try limiting the amount of RAM shadow-cljs uses by setting :jvm-opts ["-Xmx2G"]
(or 1G
) in project.clj
Can confirm -- setting :jvm-opts ["-Xmx1G"]
in project.clj :profiles -> :dev
completely fixes it. Doesn't seem to matter if it gets 1G or 2G.
(FYI to anyone later reading this, tried setting it in shadow-cljs.edn with no effect, so make sure you set it in project.clj)
still would like to know why it runs out of memory when running in cider but not otherwise though 😛
@thheller Thanks for the help. I was quite baffled by this for the last week -- I recall that you recommended bumping memory a few days ago, but I thought the cause was something else 😅
1G is usually enough unless you have really big builds then :advanced
compilation might require more
yeah, that I'm not sure of. only guess is the extra CIDER dependencies increasing mem-usage past some critical threshhold
So I was running into something similar, when a I would run shadow-cljs watch app
without the server running and I would get the attached error. This ran fine if I started the server ahead of time. Sure enough limiting the RAM helped. I have no clue why running this with more RAM causes this to break. If I run the command while providing 4GB to Java it will cause the same issue. (I'm sitting at 6GB in use out of 16GB right now, so it's not running out of memory)
EDIT: Just to be clear, this issue is FIXED by adding :jvm-opts ["-Xmx2G"]
to my shadow-cljs.edn file. I was just wondering if there was any reason why that fix works?
on CircleCI reaching a certain amount of memory caused the OOM Killer from the OS to kill the process
something from the OS must be doing that. the JVM doesn't kill itself and neither does shadow-cljs
I'd be very curious to find out what is happening but dunno where to look and this has never happened to me either
Did some more testing.
If I set :jvm-opts ["-Xmx4G"]
in project.clj, it gives the shutting down..
start message at least 30%+ of the time.
Values of 1G or 2G seem 100% stable.
Setting :jvm-opts
in shadow-cljs.edn seems to have no effect regardless of 1G, 2G, or 4G
No idea why this is -- but here's a standing offer to help troubleshoot if you get any ideas
did some more testing, can repro the shutdown >50% time with /usr/local/bin/npx shadow-cljs watch app
Tried to look through the OSX kernel logs but didn't notice anything suspicious. I'm out of ideas for now :man-shrugging:
(did try setting :jvm-opts
in shadow-cljs.edn to 1/2/4G values, didn't seem to make a difference)
no clue. the only thing triggering a shutdown should be either the JVM process getting killed, the node process getting killed or explicitely calling (shadow.cljs.devtools.server/stop!)
Possible that it's self killing due to requesting too many threads too quickly & getting execution denied? seems tied to OSX somehow. Running on a 2019 macbook pro.. wonder if an artificial Thread/sleep at some point in startup would "fix" this
(remembering that deleting the 5MB cljs
folder from .shadow-cljs cache made the issue go away 100% of the time for that run)
[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 242]
a subsequent run got through the [:app] Build completed (608 files, 1 compiled, 0 warnings, 4.34s)
and gave me back the [1:0]~shadow.user=>
, so seems that we did trigger a shutdown somehow
curious that it even gets to the build completed message since the shutdown is triggered somewhere in the middle
but I guess the work is already queued so it just keeps going and waits with the actual shutdown
otherwise looks fine. nothing suspicious except the shutdown getting triggered in the middle
> java -version
java version "1.8.0_221"
Java(TM) SE Runtime Environment (build 1.8.0_221-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)
no idea. you could try jdk11 or 12 maybe but I'd be very surprised if that changes anything
you could maybe try running (shadow/compile :app)
to see if that also gets terminated
the compilation process is almost identical so I presume they are all going to be shut down
but release
won't include all the re-frame-10x
code so it might survive due to using less resources
dunno why deleting cache files changes your success rate since compilation recreates the exact same state again anways
no clue what emacs does when running processes. maybe it is somehow monitoring the processes or causes it to have other resource constraints
but other people have reported issues with cider as well and limiting the memory seems to help
hi everybody! is there a place in shadow-cljs.edn to add es6 js files I want to add locally (not coming from npm). My use case is that I'm using a library that needs me to provide it objects that extends from others and I think I prefer to leave those as es6 js code instead of replicating that in clojurescript with goog.object/extend etc
thanks a lot!
So I was running into something similar, when a I would run shadow-cljs watch app
without the server running and I would get the attached error. This ran fine if I started the server ahead of time. Sure enough limiting the RAM helped. I have no clue why running this with more RAM causes this to break. If I run the command while providing 4GB to Java it will cause the same issue. (I'm sitting at 6GB in use out of 16GB right now, so it's not running out of memory)
EDIT: Just to be clear, this issue is FIXED by adding :jvm-opts ["-Xmx2G"]
to my shadow-cljs.edn file. I was just wondering if there was any reason why that fix works?
I want to reduce the bundle, in my case now it's about 370kb but use on not so good network
hmm yeah thats a bit large. did you create a build report? https://shadow-cljs.github.io/docs/UsersGuide.html#_build_report
cljs/core.cljs 232.04 KB cljs/pprint.cljs 98.44 KB cljs/analyzer.cljc 95.13 KB cljs/spec/alpha.cljs 54.73 KB cljs/spec/gen/alpha.cljs 23.27 KB cljs/spec/test/alpha.cljs 12.15 KB cljs/stacktrace.cljc 5.56 KB cljs/reader.cljs 3.46 KB
if you use the web UI (default at http://localhost:9630)
pprint unfortunately isn't compatible with DCE in the way it is currently written https://clojure.atlassian.net/browse/CLJS-2885
select cljs.analyzer and it'll tell you why it is included (which namespace included it)
reitit.coercion.spec -> spec-tools.core -> spec-tools.impl -> cljs.analyzer.api -> cljs.analyzer
https://code.thheller.com/demos/build-report/huge.html example report has multiple
In my dev build, I have another module called devcards, main
and devcards
both depends on base
a good rule to remember for :modules
is that you should only use them if and only if ONE user may end up using ALL modules
But If I make it another build, I know I can start two build in shadow, but Can I connect to two browser environment in one cljs repl?
You know, cider (basically except cursive) rely on repl to provide auto-completion and many other features.
need to do something like this https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/core.cljc#L24
I guess they are doing this to support the self-hosted stuff but even then it seems wrong
depends on how much code you load I guess. from your build-report it doesn't seem too excessive. but dunno how much dev stuff you add (eg. re-frame-10x adds a whole lot)
I copy that file change that macro to always return nil. I think it won't affect anything in my project
I was trying to use module split in my project, and it works pretty good. however it won't load module in a browser, (that is our mainly target browser), no tools or any information to debug. So have to give it up.
The result is good to me now, it's gizp'd 300kb. I'm going to remove a few more things. thanks for the great help today!!
shadow-cljs is a great tool & the level of support thheller provides here is exceptional. If you agree, please consider subbing https://www.patreon.com/thheller/overview to keep it rolling 🙂
;; shadow-cljs configuration
{:source-paths
["src/dev"
"src/spill"
"src/test"]
:dependencies
[[org.clojure/core.async "0.4.500"]
[reagent "0.8.1"]
[re-frame "0.10.6"]
[bouncer "1.0.1"]
[com.andrewmcveigh/cljs-time "0.5.2"]
[clj-commons/secretary "1.2.4"]
[venantius/accountant "0.2.4"]
[binaryage/oops "0.7.0"]
[binaryage/devtools "0.9.10"]
[day8.re-frame/re-frame-10x "0.3.7-react16"]
[day8.re-frame/tracing "0.5.1"]]
:builds {:app {:output-dir "web/root/js/shadow"
:asset-path "js/shadow"
:target :browser
:closure-defines {"re_frame.trace.trace_enabled_QMARK_" true}
:preloads [devtools.preload day8.re-frame-10x.preload]
:modules {:app {:init-fn spill.app.core/init}}
:main "spill.app.core"
:devtools {:http-root "web/root"
:http-port 8080}
:compiler-options {:optimizations :none}}}}
I think moving :preloads
into :devtools
, and adding re-frame.trace/trace-enabled? true
to your :closure-defines
map may do it
@thheller @stefan.age thank you very much it's working now 🙂
Is it possible to configure something like an :asset-path
to go along with [:devtools :watch-dir]
? Watch-dir does not work for me, and I think it is because the URLs are not generated like :watch-dir
expects (just /some/path
under :watch-dir
) - in my case I have a prefix.
I have this in my HTML: /f/css/front_dev.css
, and in [:devtools :watch-dir]
I have this: resources/public/css
So I think this code won't trigger: https://github.com/thheller/shadow-cljs/blob/1d3ad7ce422fde4d595868b0e9ed8ade94ee175e/src/main/shadow/cljs/devtools/client/browser.cljs#L166-L190
It just makes the hosting easier. For example for the main build, I have this:
:output-dir "resources/public/js/compiled"
:asset-path "/f/js/compiled"
and therefore that part worksthe system would expect you to use :watch-dir "resources/public"
and then put the actual file into resources/public/css/front_dev.css
and use /css/front_dev.css
as the path in HTML
yea, but that is because there is no equivalent of the asset path for the watch dir, though
its just a "virtual directory" on the web server that hosts everything under "resources/public"
It is kind of a routing simplification, since this is clojurescript being added to a legacy application
sorry I don't follow how that plays into anything? why not just create resources/public/f/css/front_dev.css
? and have you webserver serve resources/public
like every web server out there?
I can't think of an easy way to do this right now. I don't want to overcomplicate the config
I could maybe add a secondary attribute to the <link
tag itself that is used to compare
right now it only checks the href
but it could also check something like <link rel="stylesheet" href="/f/css/front_dev.css" data-css-path="/css/front_dev.css" />
or some other html attribute that lets you specify which actual file this is supposed to be