Fork me on GitHub
#shadow-cljs
<
2019-10-10
>
flowthing07:10:58

I've read the relevant bits of the user guide, but I don't really understand how to set up CSS reloading. My CSS file is at resources/public/css/main.css, and my HTTP server serves it at /assets/css/main.css. What do I need to set as the values of :watch-dir and :watch-path so that CSS reloading works?

ouvasam09:10:39

I think this should work :watch-dir "resources/public"

thheller09:10:04

should be :watch-dir "resources/public" :watch-path "/assets"

flowthing09:10:33

That works, many thanks!

Lucy Wang12:10:08

Hi guys, I'm experiencing constantly the REPL server crash after a while

$ shadow-cljs watch app
shadow-cljs - config: /path/to/shadow-cljs.edn  cli version: 2.8.61  node: v10.16.3
..... (after a while ) ....
[:app] Compiling ...
[:app] Build completed. (68 files, 1 compiled, 0 warnings, 0.04s)
shutting down ...
Worker shutdown.
Exception in thread "async-dispatch-7" java.lang.Error: java.net.SocketException: Socket closed
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1155)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Socket closed
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        at nrepl.transport$bencode$fn__5929.invoke(transport.clj:116)
        at nrepl.transport.FnTransport.send(transport.clj:41)
        at nrepl.middleware.print$send_nonstreamed.invokeStatic(print.clj:159)
        at nrepl.middleware.print$send_nonstreamed.invoke(print.clj:138)
        at nrepl.middleware.print$printing_transport$reify__6306.send(print.clj:174)
        at nrepl.middleware.caught$caught_transport$reify__6341.send(caught.clj:58)
        at shadow.cljs.devtools.server.nrepl_impl$send.invokeStatic(nrepl_impl.clj:32)
        at shadow.cljs.devtools.server.nrepl_impl$send.invoke(nrepl_impl.clj:19)
        at shadow.cljs.devtools.server.nrepl_impl$worker_exit.invokeStatic(nrepl_impl.clj:49)
        at shadow.cljs.devtools.server.nrepl_impl$worker_exit.invoke(nrepl_impl.clj:45)
        at shadow.cljs.devtools.server.nrepl_impl$repl_init$fn__6916$state_machine__2842__auto____6941$fn__6943.invoke(nrepl_impl.clj:207)
        at shadow.cljs.devtools.server.nrepl_impl$repl_init$fn__6916$state_machine__2842__auto____6941.invoke(nrepl_impl.clj:205)
        at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:973)
        at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:972)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:977)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:975)
        at clojure.core.async.impl.ioc_macros$take_BANG_$fn__2860.invoke(ioc_macros.clj:986)
        at clojure.core.async.impl.channels.ManyToManyChannel$fn__655.invoke(channels.clj:265)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        ... 2 more

Lucy Wang12:10:34

I got this for both :target :node-script and :target :browser. The shadow-cljs watch app command is still running there though.

Lucy Wang12:10:19

is this a known bug?

thheller12:10:54

@wxitb2017 are you on OSX by any chance?

thheller12:10:35

hmm yeah multiple people have been reporting this on OSX. it seems like something in the OS is starting to kill the shadow-cljs process when it reaches a certain memory usage

thheller12:10:07

you can set :jvm-opts ["-Xmx1G"] to limit it to 1gig of RAM (either in shadow-cljs.edn or project.clj in case you use lein)

thheller12:10:22

that has fixed it for everyone so far

thheller12:10:38

I still haven't figured out why it is getting killed by the OS

Lucy Wang12:10:21

But the "shadow-clj watch app" process is still there.

thheller12:10:48

hmm?

shutting down ...
Worker shutdown.
means the shadow-cljs instance shut down

Lucy Wang12:10:17

ah, so the "shadow-cljs watch app" is a node process, which wraps the java process, right?

thheller12:10:32

yeah exactly

Lucy Wang12:10:53

I see, makes sense

Lucy Wang11:10:44

Ok, I attached a cursive debugger to the shadow-cljs java process and added a breakpoint, and it seems it's the health check from java->npm that leads the worker shutdown

thheller11:10:27

so the node process is getting killed?

Lucy Wang11:10:37

nope, it's still there

Lucy Wang11:10:55

I guess there's some subtle problem of the health check loop

thheller11:10:58

oh do you maybe have a firewall or something that could be blocking the TCP socket?

Lucy Wang11:10:58

maybe only on OSX

thheller11:10:49

can you turn on :log {:level :debug} in shadow-cljs.edn? the exception should be logged and might provide a clue?

Lucy Wang11:10:52

firewall? but the bug is non-deterministic here

Lucy Wang11:10:02

good idea, I'll add that

thheller11:10:59

the way this works is the node process opens a TCP socket that the JVM process frequently connects to and checks if its still there

thheller11:10:22

otherwise there were problem where the JVM process kept running when the node process was killed

thheller11:10:31

it is odd that this doesn't happen immediately though

thheller11:10:53

wonder why the cursive debugger shows e = null

Lucy Wang11:10:52

yeah I was wondering that too

Lucy Wang11:10:19

btw I just changed the interval from 1s to 10s, and installed it by lein install. So far so good for the last 20minutes.

Lucy Wang11:10:57

I suggest 1) increase the interval, at least to e.g. 3s. 2) maybe only abort when failed 3 times in succession

thheller12:10:58

thanks for digging into this. I'd still rather find out what is happening in the first place before trying to work around it though. a local TCP connect every second shouldn't be a problem. maybe I'll change it to a heartbeat style and just leave the connection open for longer

Lucy Wang13:10:26

Makes sense, thx!

thheller12:10:09

OSX + lein seems to be the most common setup where this happens

Lucy Wang12:10:33

I use purely shadow-cljs

thheller12:10:58

you can also assign less or more RAM. depends on how much your machine has I guess. would be interesting to find out when it starts crashing 😛

Lucy Wang12:10:39

sure, I already applied the 1G limit and restarted it. Would report back if that helps. Thx a lot!

👍 4
Lucy Wang12:10:29

I tried "-Xmx1G" and "-Xms2G -Xmx5G" etc., none of them made any difference 😞 @thheller

thheller14:10:20

hey. did you try using less?

thheller14:10:45

so dmesg | grep -E -i -B100 'killed process'

thheller14:10:54

I'd really like to find out why the process is getting killed

thheller14:10:31

although it may not even be the OOM killer that is actually doing it

thheller15:10:05

how much actual RAM do you have btw?

Lucy Wang00:10:05

I'm using a mac, with 32GB RAM 🙂 And btw according to my traceback shared earlier, I doubt it's some code in the shadow-cljs/nrepl that crashed the java process http://paste.openstack.org/raw/783109/ (pay attention to Exception in thread "async-dispatch-7" java.lang.Error: java.net.SocketException: Socket closed )

Sean Poulter16:10:06

Does anyone know if there are docs for how to build the Shadow CLJS client-side code without optimizations? I’ve been trying to troubleshoot CSS reloading and was hoping to step through this code in the browser: - Source: https://github.com/thheller/shadow-cljs/blob/2.8.51/src/main/shadow/cljs/devtools/client/browser.cljs#L166-L199 which ends - In our app: /<CLOSURE_BASE_PATH>/shadow.cljs.devtools.client.browser.js My plan was to: - Clone the repo - Change the optimizations (???) - Run the “build all” script - Use npm link to use the <repoDir>/packages/shadow-cljs and shadow-cljs-jar packages in our app - Rebuild our app and step through the less minified code 🤞 Does that sound like it’d work?

thheller16:10:39

@sean.poulter I'm confused. there are no optimizations applied in watch or compile builds? only release builds and those won't contains the client/browser.cljs?

Sean Poulter16:10:12

It’s a watch build on the app end, yea.

Sean Poulter16:10:42

I guess that means it’d have to be modified on our end. :thinking_face:

thheller16:10:59

no, but I don't understand what you are actually trying to do

thheller16:10:11

the steps you described wouldn't do anything

thheller16:10:20

browser.cljs is compiled as part of your project

thheller16:10:30

if you want to use the chrome debugger to debug something in it

thheller16:10:40

open the "Sources" tab in the chrome devtools

thheller16:10:02

select the /js/cljs-runtime/shadow/cljs/devtools/browser.cljs and set a breakpoint

thheller16:10:39

you can just take the browser.cljs and add it into one of your :source-paths if you want to make modifications

Sean Poulter16:10:55

I’ve got that far. It’s minified and a bit confusing without a source map.

thheller16:10:08

thats not the .cljs file

thheller16:10:18

it also isn't minified, it is compiled .js code

👍 4
Sean Poulter16:10:35

Thanks for correcting me on that, yes.

thheller16:10:38

it should be showing those like this

thheller16:10:24

there might be .js variant of that namespace and a .cljs variant

thheller16:10:56

might be quicker if you just describe the problem you are seeing 😛

Sean Poulter16:10:36

Ah. We must have source maps off. It’s a bit different on our end and just the compiled JS under (no domain):

Sean Poulter16:10:06

Thanks for your patience with this. I’ve dropped into the Clojure world from JavaScript. 😓

thheller16:10:26

it is normal that things might be listed twice

thheller16:10:08

you can't even turn off source maps in development so unlikely they are off

lilactown16:10:14

tangent: how would I turn on source maps in release?

thheller16:10:26

:compiler-options {:source-map true}

❤️ 4
thheller16:10:21

@sean.poulter there isn't much to CSS reloading, it is watching a directory via either :watch-dir or the http root. lets say public. You have a public/css/foo.css. it will attempt to reload /css/foo.css. you can specify :watch-path "/x" which would make it try to reload /x/css/foo.css

Sean Poulter16:10:49

So the watch-path is prepended to the link href?

thheller16:10:02

it is prepending :watch-path to the PATH of the .css file it is trying to reload yes

thheller16:10:12

but if the <link doesn't already use that path, it won't reload because it will think it is a different file

👍 4
Sean Poulter16:10:16

I'll give that a try. We're using a proxy to use local code against a prod backend which is at server.localhost:port/extra-path/css/foo.css.

Sean Poulter16:10:49

I gave it a try and it didn't work, so I've gone down a rabbit hole trying to figure out why.

thheller16:10:23

uhm how is CSS reloading going to work then?

thheller16:10:33

it can't reload CSS from a different server?

Sean Poulter16:10:40

("I gave it a try" meaning, trying to follow the docs with the watch dir and watch path not working.)

Sean Poulter16:10:27

The JS hot reloads and its served from the same folder.

thheller16:10:35

if you have <link rel="stylesheet" href=""> it is never going to reload that CSS

Sean Poulter16:10:41

It's honestly quite confusing to me.

Sean Poulter16:10:12

It's a <link href="/css/foo.css"/>.

thheller16:10:20

ok. that should be fine then?

thheller16:10:40

unless you proxy just proxies that directly to the other server?

Sean Poulter16:10:55

Should be! Oooh. :thinking_face:

thheller16:10:20

all CSS reloading does is perform the request again

Sean Poulter16:10:24

It's loading our local CSS, just not reloading it.

Sean Poulter16:10:00

Yea. So all of this is why I was trying to figure out how to step through where it's deciding it's not working.

Sean Poulter16:10:47

Thanks so much for talking it through! I'll go dig into the watch-path and also figure out my source maps. :)

thheller16:10:04

just don't look under (no domain)? I only have the .js files there too

thheller16:10:04

but to repeat: you have <link href="/css/foo.css"/>. that means it will only reload files that have :watch-dir + /css/foo.css, so if :watch-dir "public" then public/css/foo.css

👍 4
thheller16:10:05

check the browser console if an actual request is triggered. you might just get the old css back from the server?

Sean Poulter16:10:32

I'll report back when I figure it out. It worked beautify when I configured the example in the README to build to dist. It doesn't make the request or log to the console. From the compiled code it looked like it was a path problem.

thheller16:10:04

you can watch the websocket messages in the chrome devtools

thheller16:10:36

{:type :asset-watch, :updates #{"/css/foo.css"}}

thheller16:10:55

if that path doesn't match your link path then nothing happens

👍 4
Sean Poulter14:10:50

Just to follow-up: 1. Thank you again! 👏 2. Watching the messages on the web socket helped. 3. The CSS reloading works great just with :watch-path "/foo/bar". I think my trouble was caused by missing the leading / on the :watch-path. :man-facepalming: 4. It also needed to have :watch-dir "dist" ’cause we weren’t using the default public folder.

tony.kay18:10:11

@thheller is there a way to figure out how a particular thing is getting into the output? I’m working on native in a shared source project and react-dom keeps leaking in…really painful figuring out how

tony.kay18:10:29

when I grep the output it isn’t obvious

tony.kay18:10:02

some dump of the dep graph

thheller18:10:50

@tony.kay the web UI has a thing. after you hit comile or watch there is a "Build Namespaces" select box

thheller18:10:00

select any namespace and it'll tell you how it ended up in the build

thheller18:10:21

the JS deps you'll find under shadow.js.shim.*

thheller18:10:57

really need to make that interface better and create a REPL fn for it or so 😛

👍 4
flyboarder18:10:21

@thheller trying to import a custom class, getting a type error TypeError: Cannot read property 'prototype' of undefined

flyboarder18:10:34

const { AuthenticationService, JWTStrategy } = require('@feathersjs/authentication');
const { OAuthStrategy } = require('@feathersjs/authentication-oauth');

class AzureB2CStrategy extends OAuthStrategy {
  constructor(...args) {super(...args);}
  authenticate({error}, params) {
    super.authenticate(error, params);
  }
}

module.exports = AzureB2CStrategy

flyboarder18:10:56

Then in my require ["./azureb2c.js" :as AzureB2CStrategy]

thheller18:10:20

look ok to me, did you check if your requires are correct?

thheller18:10:03

OAuthStrategy might just be nil/undefined?

flyboarder18:10:27

Yeah I was able to confirm this works ["@feathersjs/authentication-oauth" :as auth] then (def OAuthStrategy (obj/get auth "OAuthStrategy"))

thheller18:10:54

is this node or the browser?

thheller18:10:20

did you try :compiler-options {:output-feature-set :es7}?

thheller18:10:31

I think even :es8 might be fine for node

flyboarder19:10:07

I’ll try with that again, I was using :es6

thheller19:10:17

hmm no that wouldn't make a diff then

thheller19:10:56

are you sure you are looking at the correct thing?

flyboarder19:10:27

yeah the stack trace is pointing to the super in constructor

thheller19:10:51

did you check the compiled output?

flyboarder19:10:02

where should I look?

thheller19:10:21

for node builds it is in .shadow-cljs/builds/<the-id>/dev/out/cljs-runtime/*.js

flyboarder19:10:24

no cljs-runtime folder

flyboarder19:10:30

I could be on an old version

thheller19:10:02

no that hasn't changed in years

thheller19:10:08

did you specify a custom :output-dir? or is it not :node-script?

thheller19:10:02

for :npm-module its just your :output-dir

flyboarder19:10:21

ah yes I did

thheller19:10:52

thats not the js file?

thheller19:10:28

the js file should start with module$...

thheller19:10:48

the file you posted is the degree9.auth.azureb2c ns

thheller19:10:38

that doesn't look like :es6 output

thheller19:10:51

oh wait .. that may have the same caching bug

thheller19:10:04

try wiping the .shadow-cljs/builds cache

thheller19:10:18

it may not invalidate properly if you change :output-feature-set?

flyboarder19:10:29

kk i’ll try that

thheller19:10:17

shadow$provide.module$degree9$auth$azureb2c = function(
  global,
  process,
  require,
  module,
  exports,
  shadow$shims
) {
  require("shadow.js.shim.module$$feathersjs$authentication");
  var OAuthStrategy = require("shadow.js.shim.module$$feathersjs$authentication_oauth")
    .OAuthStrategy;
  global = function(args) {
    for (
      var $jscomp$restParams = [], $jscomp$restIndex = 0;
      $jscomp$restIndex < arguments.length;
      ++$jscomp$restIndex
    )
      $jscomp$restParams[$jscomp$restIndex - 0] = arguments[$jscomp$restIndex];
    return (
      OAuthStrategy.apply(
        this,
        $jscomp.arrayFromIterable($jscomp$restParams)
      ) || this
    );
  };
  $jscomp.inherits(global, OAuthStrategy);
  global.prototype.authenticate = function($jscomp$destructuring$var2, params) {
    OAuthStrategy.prototype.authenticate.call(
      this,
      $jscomp$destructuring$var2.error,
      params
    );
  };
  module.exports = global;
};

thheller19:10:05

thats the compiled code, definitely polyfilled which it shouldn't do when using :es6+

thheller19:10:25

dunno which JS version added ...args, it node supports that natively bump it to whatever language version that is 😛

thheller19:10:43

hmm no cache is definitely invalidated correctly if :output-feature-set changes

flyboarder20:10:06

@thheller this is with es-next as output-feature-set

flyboarder20:10:30

Same type error

flyboarder20:10:03

So it seems that any use of super in my js files fails

flyboarder20:10:33

Ok so it looks like the files are in TypeScript that I am trying to require via the JS, npm has the compiled js tho

thheller22:10:55

@flyboarder I think you are just not doing the config properly? when I set :es6 I get

shadow$provide.module$demo$test = function(global, require, module, exports) {
  require("shadow.js.shim.module$$feathersjs$authentication");
  ({
    OAuthStrategy: global
  } = require("shadow.js.shim.module$$feathersjs$authentication_oauth"));
  class AzureB2CStrategy extends global {
    constructor(...args) {
      super(...args);
    }
    authenticate({ error }, params) {
      super.authenticate(error, params);
    }
  }
  module.exports = AzureB2CStrategy;
};
//# sourceMappingURL=module$demo$test.js.map

thheller22:10:43

dunno how to check if that actually works but you are definitely not doing the config correctly if you still gets the transpiled version afterwards

Lucy Wang12:10:29

I tried "-Xmx1G" and "-Xms2G -Xmx5G" etc., none of them made any difference 😞 @thheller