Fork me on GitHub
#shadow-cljs
<
2018-01-20
>
denik02:01:36

$ lein with-profile +cljs run -m shadow.cljs.devtools.cli
=>
exception in thread "main" java.lang.UnsupportedClassVersionError: shadow/build/closure/ErrorCollector has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0, compiling:(shadow/build/classpath.clj:1:1)
ideas?

denik02:01:59

I’ve already upgraded to java 9 and downgraded again

theasp04:01:33

Is it possible to override the hot reload WS URL?

theasp04:01:17

I want to use a reverse proxy

thheller08:01:38

@theasp by default the the websocket uses the url of the page itself + the default port of 9630

thheller08:01:09

so if you serve the page over the proxy and also proxy 9630 that may work?

thheller08:01:29

@denik hmm I switched to Java9 yesterday. are you on the latest release? maybe that broke older versions which it shoudln’t have

thheller08:01:14

@denik I just published [email protected] which has class file version 52 again. (actually fixed in [email protected], sorry about that)

larshelg12:01:50

I'm trying out shadow-cljs and its working great so far. However I'm having problems getting the repl working correctly with cursive. I connect to the nrepl using cursive remote configuration. When i try to load a empty namespace its working ok, but when I load a simple namespace with some requires i get: CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec

thheller12:01:43

@larshelg welcome. that sounds like you are maybe loading a ns with (:require ["a-string" ...]) as clojure code instead of cljs?

thheller12:01:25

switching between clj/cljs is bit cumbersome in cursive

larshelg12:01:03

Hey, thanks for the welcome 🙂

larshelg12:01:39

Well I start up the repl in cursive, the connection seems fine.

larshelg12:01:54

Connecting to rempte nREPL server....

larshelg12:01:02

Clojure 1.9.0

larshelg12:01:23

then i load a simple namespace

larshelg12:01:14

i have to switch from clj to cljs in the repl config bar

thheller12:01:20

when you connect to Cursive the REPL starts out of CLJ. so everything you eval is eval’d as clojure

thheller12:01:42

you then need to start a build “watch”

larshelg12:01:55

but i thought it would be enough to switch to cljs in the toolbar....

thheller12:01:01

either via the command line shadow-cljs watch the-build or via the REPL itself (shadow.cljs.devtools.api/watch :the-build)

thheller12:01:25

you then need to switch the nREPL connection itself over to CLJS by selecting which build you want to talk to

thheller12:01:35

(shadow.cljs.devtools.api/nrepl-select :the-build)

thheller12:01:43

at this point the eval happens in CLJS

thheller12:01:56

switching the cursive CLJ/CLJS option only controls the editor part

thheller12:01:04

it does not interact with the REPL connection itself

larshelg12:01:08

i see, i'll try that then 🙂

larshelg12:01:54

i can sart the cursive repl before the shadow-cljs watch the-build?

larshelg12:01:17

i thought i had to start the watch build first...

thheller12:01:48

shadow-cljs has a server mode which will start the nrepl server

thheller12:01:07

one way to start the “server” is running shadow-cljs watch the-build

thheller12:01:17

that starts the server AND watches the build

thheller12:01:27

you can also just run shadow-cljs server or shadow-cljs start

thheller12:01:42

that just starts the server and lets you do everything else independently

larshelg12:01:11

i see, so far i only started with shadow-cljs watch "build"

thheller12:01:32

that is ok. then the build is already running

thheller12:01:43

(shadow.cljs.devtools.api/nrepl-select :the-build) would be enough then

thheller12:01:11

the nrepl can also completely control the server, that is why it starts out as clojure

thheller12:01:53

colin is working on a REPL rework for Cursive which hopefully makes things a little easier to work with

larshelg12:01:57

After running the (shadow.cljs.devtools.api/nrepl-select :server) the repl still complains about not being able to load clojurescript in a clojure repl

larshelg12:01:25

I then try to change to clojurescript in the toolbar and load the namespace

thheller12:01:44

try something simple like (js/alert "foo")

larshelg12:01:11

the namespace loades ok, so thats good

larshelg12:01:30

but when i do a (in-ns 'shadowpusher.core)

larshelg12:01:39

i get Thes i no connected JS runtime

larshelg12:01:03

which seems weird, since the watch build is running, and everything else seems fine...

larshelg12:01:32

yeah the alert gives same thing "no connected js runtime"

thheller12:01:54

no connected runtime means that you have not yet loaded the page in the browser

thheller12:01:01

when building for the browser

larshelg12:01:07

its a node build...

thheller12:01:10

or started the node process

thheller12:01:28

if you connect to a build you are responsible for running it

larshelg12:01:37

aha yeah i havent started the node process..

thheller12:01:53

you can run shadow-cljs node-repl and (nrepl-select :node-repl)

thheller12:01:10

that starts a node process automatically without a specific build

thheller12:01:03

but builds do not start any node processes

larshelg12:01:05

its all working now, thanks for the help @thheller

denik14:01:52

@thheller yup downgrading shadow-cljs worked. Will try .137. Thanks!!

theasp15:01:07

@thheller figwheel allows me to set :websocket-url to <wss://example.com/figwheel>, and the proxy sends that URL to figwheel. This way I don't need to make random ports public and can control access in one spot. I'd like to do the same with shadow-cljs

thheller15:01:10

sure I can add that

thheller15:01:48

there is just one issue

thheller15:01:06

should be easily solved if you proxy correctly though

thheller15:01:12

the websocket url takes parameters

theasp15:01:43

I don't think they are affected

theasp15:01:57

Sente uses paramaters too

thheller15:01:18

well the parameters are in the path

theasp15:01:27

Ohh! Yeah, no problem

thheller15:01:30

can’t remember why I used the path instead of query args

thheller15:01:08

either way if you set :websocket-url "" and forward /shadow-cljs/* to host:9630 that should be fine

thheller15:01:38

(it also does some XHR requests for file reloading)

thheller16:01:10

@theasp [email protected] lets you configure :devtools-url "". must be http or https, it will use ws where approriate.

thheller16:01:45

as long as /* is forwarded to the actual shadow-cljs instance that should work

thheller16:01:04

what are you using as proxy?

thheller16:01:22

:builds {:build {:devtools {:devtools-url ...}}}} that is

thheller16:01:53

thinking about it I could probably move that to the top level to make things easier

theasp17:01:00

Awesome! TYVM, I'm using nginx. I can give you an example if you want one

denik17:01:36

@thheller trying to eval the ns form into the repl I get

CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec:
In: [1] val: ((:require [rum.core :as rum] ["d3" :as d3])) fails spec: :clojure.core.specs.alpha/ns-form at: [:args] predicate: (cat :docstring (? string?) :attr-map (? map?) :clauses :clojure.core.specs.alpha/ns-clauses),  Extra input
 #:clojure.spec.alpha{:problems [{:path [:args], :reason "Extra input", :pred (clojure.spec.alpha/cat :docstring (clojure.spec.alpha/? clojure.core/string?) :attr-map (clojure.spec.alpha/? clojure.core/map?) :clauses :clojure.core.specs.alpha/ns-clauses), :val ((:require [rum.core :as rum] ["d3" :as d3])), :via [:clojure.core.specs.alpha/ns-form], :in [1]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x1f563fb3 "clojure.spec.alpha$regex_spec_impl$reify__2436@1f563fb3"], :value (dashboard.core (:require [rum.core :as rum] ["d3" :as d3])), :args (dashboard.core (:require [rum.core :as rum] ["d3" :as d3]))}, compiling:(/Users/den/Dropbox/dev/code/centriq/centriq-automation/transcription/dashboard/src/dashboard/core.cljs:1:1) 

denik17:01:53

it’s because of the string "d3"

denik18:01:07

^ this is using proto-repl

theasp19:01:46

@thheller DEVTOOLS: REPL init successful 🙂

thheller20:01:03

@denik that looks like it might be trying to eval in clojure still. clojure doesn’t support the strings.

thheller20:01:33

I don’t know much about proto repl

thheller20:01:14

hmmm just did a quick test. it seems to connect fine but I can’ get it to eval anything

thheller20:01:31

it sends a bunch of stuff it shouldn’t be sending 😞

cjsauer20:01:44

When using the :npm-module target (https://shadow-cljs.github.io/docs/UsersGuide.html#target-npm-module), is it possible to somehow embed a REPL server into that outputted JS file? I'm trying to embed it into a create-react-app application, and then connect to it from CIDER :thinking_face:

thheller20:01:51

yes but you need to shadow.cljs.devtools.client.node ns somehow

thheller20:01:02

since :npm-module doesn’t have clearly defined entries

thheller20:01:47

require("shadow-cljs/shadow.cljs.devtools.client.node") should do it if you are using the default :output-dir

thheller20:01:55

eg. in the generated index.js

thheller20:01:40

create-react-app makes things a bit tricky since its using ES6 import which can’t be conditional

thheller20:01:59

otherwise I’d recommend wrapping the require in the usual process.env.NODE_ENV check

cjsauer20:01:09

I'm actually setting :output-dir to node_modules/shadow-cljs

cjsauer20:01:17

To gain access to it using require

thheller20:01:22

I’d recommend just rewriting the index.js to require

thheller20:01:13

it has been a while since I messed with CRA

cjsauer20:01:31

You mean add require("shadow-cljs/shadow.cljs.devtools.client.node") to App.js?

thheller20:01:41

thats what I did back then

thheller20:01:25

oh wait right .. not the node thing. require("shadow-cljs/shadow.cljs.devtools.client.browser");

cjsauer20:01:58

Those links are great, thank you. I see you also set :runtime to :browser

cjsauer20:01:10

In the shadow-cljs.edn file

thheller20:01:25

CRA was the first project I used as a proof of concept for the integration

thheller20:01:48

May 21, 2017 … lots has changed since then

thheller20:01:32

and the live reloading of CRA often completely messed up the REPL

thheller20:01:45

not sure if that situation improved on the CRA side

justinlee20:01:09

is there a doc somewhere that explains what shadow-cljs is doing? you mentioned transpilation in the other thread. i’ve just gotten my head around why we have externs and how to avoid them with cljs-oops but it sound like shadow is doing something else. (the docs on how to use shadow are great, btw, kudos, i just like to have an idea of what the tool is actually doing)

thheller20:01:55

wrote a whole series about Js dependencies here https://code.thheller.com/

justinlee20:01:24

nice thanks. this looks like it was written for me 🙂

cjsauer20:01:13

@thheller once I have shadow.cljs.devtools.client.browser loaded, how would I start a REPL client? Something like: shadow-cljs cljs-repl?

cjsauer21:01:07

Hmm...I keep getting this:

[3:1]~cljs.user=> (js/alert "test")
There is no connected JS runtime.

cjsauer21:01:48

I tried running shadow-cljs cljs-repl dev (my build id is "dev")

thheller21:01:31

yeah that means that the browser did not connect

thheller21:01:47

you should see a connect message when you open the browser console

cjsauer22:01:35

Hm...still no luck. Once I have required shadow.cljs.devtools.client.browser in index.js, is there any function I need to call to actually "start" it? Or is the act of requiring it enough?

thheller22:01:49

yeah .. its completely broken

thheller22:01:35

just fixed that part but the rest still doesn’t work

cjsauer22:01:56

Sounds good. Thanks for the help @thheller

thheller22:01:00

couldn’t find an easy fix.

cjsauer22:01:31

I wish I understood this sort of stuff better so I could help out 😕 Compilers are still kind of magic to me haha

thheller22:01:15

any particular reason you want to use create-react-app?

thheller22:01:34

:npm-module works ok but the REPL is never going to be reliable anyways given the amount of stuff webpack does to the code

cjsauer22:01:03

Well, my team is primarily JS guys, and they all use create-react-app. I'm looking for a good way to introduce ClojureScript to the team, but in a really seamless way that doesn't interrupt their workflow too much. I figured if I could at least jack a REPL in, I'd be able to preserve both workflow styles pretty well...

cjsauer22:01:26

shadow-cljs seems like a really promising tool for integration projects like the ones we face

thheller22:01:08

yeah thats what :npm-module was supposed to achieve, which it sorta does until you launch a REPL 😉

cjsauer22:01:53

hehe, I'm excited to see it working. There's some really interesting workflows hiding in the JS/CLJS pairing

thheller22:01:02

REPL definitely works way better when shadow-cljs is in control and its JS doesn’t get rewritten by something else

thheller22:01:57

we’ll see if I can fix it tomorrow. I’m not actually sure it ever worked properly beyond basic (js/alert "foo") 😉

cjsauer23:01:45

cool, I'll keep an eye on the repo :thumbsup:

thheller21:01:24

@lee.justin.m I can go into technical details if you’d like. I mostly left those out and focused more on the “why”

thheller21:01:38

but in essence it is doing the exact same thing webpack and others are doing

justinlee21:01:52

@thheller My basic curiosity is that I’m a javascript programmer. Things like webpack and minification and babel all screw with the code and seem to be able to do so without breaking everything. So far I’m a little curious and terrified that certain kinds of innocuous code (.field js/thing) can be broken so easily. How do other tools do it? So my interest in (1) the practical, which so far has been: use cljs-oops for all external accesses and (2) learn about different approaches people are taking. It seems like you are doing something different in shadow to make interop safer (?). At any rate I just have a general technical curiosity here. No specific burning need.

thheller21:01:35

I don’t recommend using cljs-oops at all. shadow-cljs has greatly improved externs inference support which has elimitated the need for almost every extern I used before

thheller21:01:34

basically I run everything with :infer-externs :auto these days

thheller21:01:42

very occasionally I add a typehint

thheller21:01:57

things like (.field js/thing) are fully inferred

justinlee21:01:33

Regarding this: We only need externs for the code we actually call from CLJS, not everything the JS code uses. That suggests that over-specifying externs keeps code in that might otherwise be elided. Does GCC do DCE on npm modules? I thought the whole idea is that the compiler won’t do advanced optimizations on npm modules and the externs are a directive to the compiler not to name mangle those symbols in optimized code.

thheller21:01:06

no, shadow only does :simple for everything on node_modules.

thheller21:01:23

but the CLJS/goog parts still run through :advanced

thheller21:01:37

so at the interop points you need externs

thheller21:01:58

but since shadow now knows more about those interop points due to the requires in ns and such

thheller21:01:05

it can do better externs inference for those

justinlee21:01:09

I was just trying to figure out why exactly too many externs increases code size.

thheller21:01:39

the increase is minimal

thheller21:01:56

but say you have createThing defined somewhere in the externs

thheller21:01:26

the closure compiler will now protect that name whenever it can’t exactly identify which type it is on

thheller21:01:37

since CLJS is untyped that basically means always

thheller21:01:55

so createThing will never be shortened or removed, even if it could be otherwise

justinlee21:01:57

it doesn’t really matter i guess but given that the javascript module is going through :simple optimizations i was think that createThing couldn’t ever be renamed or removed anyway

thheller21:01:38

yes, only very basic code removal is done for node_modules

thheller21:01:09

but CLJS is still fully :advanced optimized so it could be removed there (in case it exists)

thheller21:01:30

the goal is to run everything through :advanced at some point

thheller21:01:43

thats what CLJS tries to do .. but it just isn’t reliable enough yet.

justinlee21:01:50

This great stuff. I’m going to read through your stuff before I ask you any more questions. 🙂

thheller21:01:11

most of it should be covered in some form

thheller21:01:31

feel free to ask though. always helps to validate that I’m not doing crazy things 😉

justinlee21:01:01

I do have one other question, which is actually about DCE. Do you know if the symbol munging thing is necessary to do DCE? I’ve often wondered if we could just turn on DCE without the symbol munging and get some stuff for free.

thheller21:01:58

the munging no but the collapse

thheller21:01:10

in CLJS (or closure) everything is namespaced

thheller21:01:24

so if you have (ns demo.app) (defn foo [])

thheller21:01:38

you get a demo.app.foo = function() {} basically

thheller21:01:04

closure collapses this down to demo$app$foo so it can remove the nested objects

thheller21:01:28

this collapse is necessary for DCE (AFAIK)

thheller21:01:04

the rename then just shortens it down to something like xA (or removes it entirely)

thheller21:01:52

the closure compiler has some options that let you fine tune what it does to the code

thheller21:01:13

disabling variable renaming or property renaming is an option

thheller21:01:38

but if you disable too much you basically end up at :simple

thheller21:01:57

which is still better then uglifyjs but nowhere near :advanced

thheller21:01:31

closure is already pretty good at optimizing ES6. it just struggles with some dynamic CJS things people do

thheller21:01:11

hopefully more JS people will begin publishing more ES6 code

thheller21:01:29

slow procress I guess if node can’t even decide how to do proper es6 interop

justinlee21:01:06

it would be interesting if you could do static analysis on a body of code to try to see if code was already safe for advanced optimization. you’d get a lot of false positives, but it might be able to show you everywhere something dangerous is happening (e.g. string property access)

thheller21:01:04

well you can always set :js-provider :closure and see if you code still works. that does :advanced for everything. if it doesn’t go back to :shadow.

thheller21:01:54

full analysis might be painful given the dynamic nature of JS

justinlee21:01:25

yea it would probably not work in a lot of cases, but it would be awesome if you could use babel or something to provide certainty that a module is safe for compilation

justinlee21:01:45

anyway, i don’t even understand what the hell is going on at any level so that’s just a pipe dream

thheller21:01:51

unfortunately babel is responsible for some of the breakage 😉

thheller21:01:49

doing what webpack and babel do however seems to be the safest way to deal with node_modules so I’m doing that

justinlee21:01:20

well i’m interested to dig in. so far the safest route for using modules for me has been to use distribution builds with no externs and run all accesses through cljs-oops.

thheller21:01:13

if you like cljs-oops by all means keep using it and reduce it gradually maybe

justinlee21:01:49

oh i’m not wedded to anything, but cljs-oops is better than relying on some half assed broken not up to date externs file that someone made 3 years ago

thheller21:01:32

better yet use :infer-externs :auto and let the compiler tell you if it doesn’t unterstand something

thheller21:01:54

whether you then use ^js typehint or cljs-oops as the fix is up to you

cjsauer22:01:35

Hm...still no luck. Once I have required shadow.cljs.devtools.client.browser in index.js, is there any function I need to call to actually "start" it? Or is the act of requiring it enough?