Fork me on GitHub
#shadow-cljs
<
2019-11-21
>
Quest05:11:36

I'm using shadow-cljs to compile react-native to an Android emulator (code is forked from https://github.com/PEZ/rn-rf-shadow ) It works fine until I encounter an error. Sometimes they disconnect the CIDER nREPL (started via Emacs M-x cider-jack-in-cljs) with the runtime in the Android VM, then all REPL commands result in:

example.repl> (js/alert "test")
REPL command timed out.
Is there a way to recover from this state? Reloading the app via CMD+M (in the emulator) doesn't re-establish the connection from Emacs to the in-process nREPL

p-himik08:11:15

Just to make sure - does only js/alert timeout or any command (given that you haven't executed js/alert before it). The thing is - js/alert is blocking. The execution will not continue till you interact with the pop-up dialog.

Quest09:11:16

any command gives the REPL command timed out unfortunately

Quest05:11:12

(note that the hot-reloading part of shadow-cljs still works fine, it's just the REPL connection that gets borked)

thheller10:11:00

the problem is that in android the react-native app does not disconnect the websocket when you reload the app

thheller10:11:11

so shadow-cljs thinks that is still connected but never gets a reply back

thheller10:11:43

I added (shadow.cljs.devtools.api/repl-runtime-clear) to get rid of "stale" connections like that but it has to be called from a CLJ REPL

Quest11:11:01

Thanks @thheller, appreciate that there's a way to do this. I wrote a section for my README.md but wanted to repost here for channel record. Recover disconnected shadow-cljs REPL You might accidentally crash your react-native app on your emulator/browser with a big red error screen. You can hit Reload to restart the app into a working state, but putting any command into your shadow-cljs REPL will time out.

myapp.core> (js/alert "test")
REPL command timed out.
To recover this you'll need to: 1) Quit the CLJS REPL 2) Tell shadow-cljs to clear its REPL connection 3) Reconnect the CLJS REPL The following commands should achieve this.
cljs.user> :cljs/quit
;; wait at least 5 seconds for session cleanup
user> (shadow.cljs.devtools.api/repl-runtime-clear)
user> (shadow/repl :app)

thheller11:11:52

note that the cleanup timeout is currently 5sec

thheller11:11:33

so if you do this to fast it might not kick the old session

Quest11:11:37

:thumbsup: & updated the note

thheller11:11:42

no the wait has to be before repl-runtime-clear

thheller11:11:54

not before opening the REPL again

Quest11:11:04

Ahh, good catch. Fixed again

thheller11:11:33

also only need to wait 5sec after reloading the actual app. so may not need to wait at all depending on how fast you do this 😉

Quest11:11:52

Hmm, my REPL never recovers from the disconnected state after hitting Reload . That's after I tank the app with something bad like (js/require "foo")

Quest11:11:09

the manual steps fix it, but haven't observed autocorrecting behavior

thheller11:11:33

dunno. gotta go right now, be back later

Quest11:11:49

No sweat, gotta sleep soon myself. If you have any recommendations I'll try them tomorrow, but thanks either way 🙂

orestis09:11:45

So @thheller I noted that you mentioned in #cljs-dev that CLJS is stuck at an older version of Google Closure compiler -- is that something that carries over to shadow-cljs too or do you override the google closure dependency?

thheller10:11:13

shadow-cljs uses the latest closure compiler and library. it does override those yes.

Filipe Silva10:11:24

I'm having some trouble using the main option for node-test targets... given this namespace:

(ns app.core-spec
  (:require [cljs.test :refer-macros [deftest is]]))

(deftest something (is true))
and config:
:test {:ns-regexp "app.core-spec"
         :output-to "out/test.js"
         :target :node-test}}
running the tests works:
$ shadow-cljs compile test && node out/test.js
shadow-cljs - config: D:\sandbox\devcards-test\shadow-cljs.edn  cli version: 2.8.69  node: v10.16.0
shadow-cljs - socket connect failed, server process dead?
[:test] Compiling ...
[:test] Build completed. (47 files, 2 compiled, 0 warnings, 2.04s)

Testing app.core-spec

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
Done in 13.20s.
but if instead I require the test runner:
(ns app.core-spec
  (:require [cljs.test :refer-macros [deftest is]]
            [shadow.test.node :as stn]))

(deftest something (is true))

(defn main []
  (stn/main))
and alter the config to use my main
:test {:main app.core-spec/main
         :output-to "out/test.js"
         :target :node-test}}
no tests will be found:
kamik@RED-X1C6 MINGW64 /d/sandbox/devcards-test (master)
$ yarn test:once
yarn run v1.17.3
$ shadow-cljs compile test && node out/test.js
shadow-cljs - config: D:\sandbox\devcards-test\shadow-cljs.edn  cli version: 2.8.69  node: v10.16.0
shadow-cljs - socket connect failed, server process dead?
[:test] Compiling ...
[:test] Build completed. (47 files, 2 compiled, 0 warnings, 1.89s)

Ran 0 tests containing 0 assertions.
0 failures, 0 errors.
Done in 12.60s.

Filipe Silva10:11:41

I also tried using both :main and :ns-regexp at the same time but that failed with a weird error

thheller10:11:44

why are you trying to override :main?

thheller10:11:38

I think the issue with par-compile might be that it sets up a circular dependency if your ns-regexp matches your main ns

thheller10:11:53

possible that I don't check that

thheller10:11:25

make sure that the "runner" doesn't match the regexp

thheller10:11:04

also do yourself a favor and run shadow-cljs server separately. waiting 12sec to a test to run is horrible

Filipe Silva11:11:27

yeah on the real project I start the server separately, I didn't do it here because I forgot

Filipe Silva11:11:52

I'm trying to override main because on my real setup there's some logic I need to run conditionally at the start

Filipe Silva11:11:17

so the intended usage is that both main and ns-regexp need to be set?

thheller11:11:50

:ns-regexp controls which namespaces are included yes

thheller11:11:40

you can leave it to something that doesn't match and include all the namespaces in the :main ns directly

thheller11:11:54

doesn't matter really

Filipe Silva11:11:28

the user guide mentions that it defaults to "-test$"

Filipe Silva11:11:50

so I thought that if I set main without setting ns-regexp it would run the tests in the included namespaces in main, including itself

thheller11:11:13

Testing app.core-spec

thheller11:11:17

that doesn't match -test?

Filipe Silva11:11:50

it doesn't, so it falls into the case you suggested: > you can leave it to something that doesn't match and include all the namespaces in the :main ns directly

Filipe Silva11:11:11

ns-regexp didn't match anything but in main there was a test

thheller11:11:24

well not really ...

thheller11:11:41

WHAT THE FUCK IS THIS NEW FUCKING SLACK SHIT

Filipe Silva11:11:55

the future is coming thheller, and it is annoying

thheller11:11:06

drives me mad ... can't type :main anymore without it trying to make it something else

thheller11:11:38

so the way this works is that :ns-regexp is responsible for adding namespaces

Filipe Silva11:11:42

it does get better though, on my work slack it started last week and everyone had the same reaction, this week no one seems bothered

thheller11:11:02

just accepting everything is shit I guess ...

thheller11:11:48

:main only controls running the tests and it loads the tests via a macro

thheller11:11:11

thats why the :main ns needs a {:dev/always true} annotation in the ns

thheller11:11:16

otherwise the macro can't update properly

thheller11:11:52

ok slack ... just don't highlight ... thats fine ...

😠 4
thheller11:11:52

might be a problem if you just delegate to the default runner ns

thheller11:11:20

totally possible this all has a bunch of issues. I basically only tested the default setup

Filipe Silva11:11:00

ok, I think I'm starting to understand a bit better... so if I want to both have my own main (for init logic), and not match any ns (because I don't have dedicated test ns), is there an approach I can take?

Filipe Silva11:11:21

you mentioned just including the ns in the main ns

thheller11:11:45

if you write your own main-ns I would suggest copying the default as is and only override what you need

thheller11:11:51

do not delegate to it

thheller11:11:02

the interaction with the macro is a bit funky

thheller11:11:26

thats the macro

Filipe Silva11:11:45

I tried these ns and it seems to run with main

(ns app.core-spec
  (:require [app.core-spec-two]
            [shadow.test.node :as stn]))

(defn main []
  (stn/main))
(ns app.core-spec-two
  (:require [cljs.test :refer-macros [deftest is]]))

(deftest something (is true))

Filipe Silva11:11:00

will try the way you just mentioned

Filipe Silva11:11:50

hmm, so the above finds and runs the single tests

Filipe Silva11:11:01

but using the ns as you said does not

Filipe Silva11:11:03

(ns app.core-spec
  {:dev/always true}
  (:require
   [shadow.test.env :as env]
   [cljs.test :as ct]
   [shadow.test :as st]))

;; FIXME: add option to not exit the node process?
(defmethod ct/report [::ct/default :end-run-tests] [m]
  (if (ct/successful? m)
    (js/process.exit 0)
    (js/process.exit 1)))

(defn main []
  (-> (env/get-test-data)
      (env/reset-test-data!))
  (st/run-all-tests))

Filipe Silva11:11:11

$ shadow-cljs compile test && node out/test.js
shadow-cljs - config: D:\sandbox\devcards-test\shadow-cljs.edn  cli version: 2.8.69  node: v10.16.0
shadow-cljs - connected to server
[:test] Compiling ...
[:test] Build completed. (46 files, 1 compiled, 0 warnings, 0.82s)



Ran 0 tests containing 0 assertions.
0 failures, 0 errors.
Done in 1.83s.

thheller11:11:27

please just out of habit STOP using the same namespace pattern for :mountain:

⛰️ 4
thheller11:11:33

ok slack ... whatever ...

thheller11:11:49

yeah thats it 😛

thheller11:11:02

horrible for clojure especially I guess

thheller11:11:18

every keyword will be a shitty emoji 😛

orestis11:11:40

HN thread (https://news.ycombinator.com/item?id=21589647) says to use /feedback to complain, let's see 🙂

thheller11:11:50

use something like app.test-runner or so for :main

👍 4
thheller11:11:32

please setup a demo repo for this .. can't look into it further right now

Filipe Silva17:11:34

set up a repro, and I think there isn't anything to look into proper

Filipe Silva17:11:49

this repo has 2 deftests

Filipe Silva17:11:59

in one of them, devcards is also imported, but not used

Filipe Silva17:11:56

running yarn test will run node-test test targets with ns-regexp, main pointing at a import of shadow.test.node, and main pointing at a copy of shadow.test.node

Filipe Silva17:11:24

both ns-regexp and main pointing at a copy of shadow.test.node work as expected and show two tests running

Filipe Silva17:11:34

main pointing at a import of shadow.test.node only shows one test running (the one in the ns that doesn't import devcards)

Filipe Silva17:11:13

I don't know why the devcards import should be relevant, but you told me to use a copy anyway so I suppose it's related

Filipe Silva17:11:21

maybe because devcards also declares a deftest

Filipe Silva11:11:53

roger, will do

Filipe Silva11:11:56

thank you for your time

Quest11:11:01

Thanks @thheller, appreciate that there's a way to do this. I wrote a section for my README.md but wanted to repost here for channel record. Recover disconnected shadow-cljs REPL You might accidentally crash your react-native app on your emulator/browser with a big red error screen. You can hit Reload to restart the app into a working state, but putting any command into your shadow-cljs REPL will time out.

myapp.core> (js/alert "test")
REPL command timed out.
To recover this you'll need to: 1) Quit the CLJS REPL 2) Tell shadow-cljs to clear its REPL connection 3) Reconnect the CLJS REPL The following commands should achieve this.
cljs.user> :cljs/quit
;; wait at least 5 seconds for session cleanup
user> (shadow.cljs.devtools.api/repl-runtime-clear)
user> (shadow/repl :app)

Eliraz15:11:51

how would I tell shadow-cljs where is my html file to take from?

Eliraz15:11:03

I mean, do I have to manually copy it to the build/target folder?

thheller15:11:34

shadow-cljs has no support for processing html, so yes you copy it manually or with another tool

Eliraz15:11:19

okay. maybe I could use webpack here to do it

Drew Verlee19:11:36

When trying to host my app, i'm noticing that the the requests by the browser triggered in the index.html to load the other assets (e.g main.js) are returning 404. I assume there is some path issue i need to resolve. I assume this is a function of the hosting service (in this case google bucket) but i wanted to ask here in case there is some configuration concept im missing. I dont think any of the production settings (e.g url hashing) would make a difference and i didnt see anything else listed.

Drew Verlee19:11:11

maybe setting homepage in the package.json?

thheller19:11:08

if the initial load already fails then your path in the html is incorrect?

thheller19:11:55

I haven't used the google stuff before. I'm assuming you upload the files somewhere so make sure you upload all the files properly?

Drew Verlee19:11:42

Thanks heller. The issue seems to be that the requested url is missing the bucket name e.g. its requesting http://storage.googleapis.com/js/compiled/main.js but the file is at http://storage.googleapis.com/<bucket-name>/js/compiled/main.js

thheller19:11:29

that means you use :asset-path "/js" and likely need "js"

thheller19:11:39

or just including the bucket-name if you know it

Drew Verlee19:11:08

because /js implies relative?

thheller19:11:23

no /js implies absolute from the domain root

thheller19:11:27

js is relative to the current file

thheller19:11:44

that'll become js/main.js loaded from index.html

Drew Verlee19:11:05

that will be it then

thheller19:11:57

if you just have one file you only need to change your HTML

thheller19:11:21

so that it loads <script src="js/main.js"> :asset-path does nothing for the initial file, only if you use multiple :modules

Drew Verlee19:11:24

we do use multiple modules.

thheller19:11:02

ok then :asset-path is relevant too. still need to change the html for the initial load though

muhanga19:11:44

Is it possible to get some sort of debug in compilation process using console flags? I am trying to debug my configuration of :source-paths ["src" "env/dev"] (ns-es in "env/dev" are ignored for some reason).

lilactown20:11:45

@muhanga do you mean that the namespaces are not found, even though the files exist on disk?

muhanga20:11:46

@lilactown Yes. I have namespace 'tst.cards' declared in "env/dev" dir and it is ignored and not taken into compilation. I have :lein true enabled, but i am not sure how those two interract.

lilactown20:11:59

what’s the file path to that namespace?

lilactown20:11:08

env/dev/tst/cards.cljs?

lilactown20:11:24

are the source paths defined in a leiningen profile?

muhanga20:11:34

@lilactown file path is env/dev/cljs/tst/cards.cljs . And source paths are defined in leiningen profile and i am able to build using lein.

lilactown20:11:53

That path is wrong. It will resolve the ns from the root of the source path

lilactown20:11:29

it’s looking in env/dev/tst/cards.cljs and not finding the file

lilactown20:11:56

if you want to nest it in that cljs directory then you need to add to the source-path instead

lilactown20:11:05

:source-paths ["src" "env/dev/cljs"]

muhanga20:11:29

Okay, this actually shed some light. Does shadow-cljs actually use :source-paths declared in lein config when :lein true is declared? Or it uses :cljsbuild\{build}\source-paths ?

muhanga20:11:14

Cause i think that anything inside :cljsbuild are actually ignored.

lilactown20:11:24

:lein true will use the source-paths defined in your leiningen config. it does not use anything to do with cljsbuild

muhanga20:11:32

Nasty. so if i want to be able to build cljs using both lein and shadow i basially need to disable :lein true

lilactown20:11:53

if you prefer to split your config and source paths between CLJ and CLJS, then what I typically do is use leiningen to manage CLJ dependencies and source paths and shadow-cljs.edn to manage CLJS dependencies and source paths

lilactown20:11:11

you can also specify a profile to use, if you prefer

dpsutton20:11:14

why do you want to be able to build with both lein and shadow?

☝️ 4
muhanga20:11:26

@lilactown Ouch. I need to read docs again. And couple of time more.

muhanga20:11:45

@dpsutton as of now i am trying to understand how shadow works so using it on existing lein project help me immensly. So i am trying to make them work in parallel. But more i try, more i think this is not really ideal approach to learning.

dpsutton20:11:12

i would branch and move over. if works PR

dpsutton20:11:55

i did exactly that and unfortunately we couldn't move over since we serve our front-end over https and local networking hyjinx but figwheel will happily connect an insecure websocket over ws not wss

thheller21:11:35

@dpsutton :devtools-url "`http://localhost:9630``"` should work and also use ws:// even if you serve the initial page over https?

thheller21:11:13

@muhanga any cljsbuild config is ignored but you can just setup regular :profiles {:dev {:source-paths ["src" "env/dev"]}} with :lein true

David Pham21:11:27

@dpsutton there are some Clojure libraries that only works with lein

dpsutton21:11:51

@thheller i'll try that this afternoon. thanks

David Pham21:11:06

I wish I could use shadow-CLJS with that :)

thheller21:11:27

kinda tough if that uses figwheel directly 😛

dpsutton21:11:29

that uses figwheel main?

dpsutton21:11:54

but i'm sure it could be ported over. just no benefit i suppose. i don't think devcards require figwheel just its a bit easier

thheller21:11:52

devcards works just fine in shadow-cljs

thheller21:11:04

but kamera has a bunch of clojure code that is talking directly to figwheel

thheller21:11:08

so that all would need to be replaced

thheller21:11:18

dunno why it is talking to figwheel at all though

muhanga21:11:27

Basically i am trying to bootstrap spring + reagent + d3 + devcards combo to shadow-cljs (with minimal changes of lein configs) and everyting expect devcards works very nice with shadow. Looks like i will need to bite the bullet and do some rewrites from scratch. Thank you all.

thheller21:11:41

which part of devcards doesn't work?

dpsutton21:11:51

devcards require a constant to be set. have you ensured that is set?

muhanga21:11:25

@thheller for some reason devcards can't find react during initialization emiting bunch of "React is not defined"

thheller21:11:53

yes, that is because devcards doesn't declare its dependency on react in any way

thheller21:11:13

so you include [cljsjs.react] and [cljsjs.react.dom] in your code somewhere before including devcards

thheller21:11:16

then it should be fine

muhanga21:11:13

Yep. It worked.

muhanga21:11:54

I am happy camper now. Thank you very much for this bits and pieces of knowledge.

David Pham21:11:26

Haha the devcard trick also took me a while to figure out

parens 4