Fork me on GitHub
#cljs-dev
<
2020-04-12
>
dnolen00:04:16

looks like things haven't changed so much w/ React, 5 dynamically referred to plugins still need to be in externs when advanced compiling

dnolen00:04:22

still same codebase as the above the following funnels everything through Closure instead

dnolen00:04:46

{:main cljs-bundle.core
 :output-to "out/main.js"
 :output-dir "out"
 :language-out :es6
 :npm-deps true
 :externs ["externs.js"]}

dnolen01:04:04

browser REPL quick start works like a charm with the bundle changes

dnolen11:04:16

hey let's not use threads, it's really annoying to see backlog

👍 4
dnolen11:04:51

optimal code splitting is a Google Closure thing - not a CLJS or shadow-cljs thing

dnolen11:04:04

neither tool can do what Closure does for Closure-aware code

dnolen11:04:16

all you can do for npm stuff is prepend it

dnolen11:04:12

but now that I think about it, the existing code-splitting feature probably already works or can be made to work trivially

dnolen11:04:57

that said if we can't - not a big deal IMO, if shadow-cljs can offer this feature and we can't it doesn't really have any bearing on the bigger goal

dnolen11:04:25

code-splitting is a great feature - but it doesn't have anything to do really w/ the ecosystem effects I'm interested in

thheller11:04:35

the issue with code-splitting is getting that to work in webpack

thheller11:04:51

because that has a fundamentally different view on how to do that then the closure-compiler does

thheller11:04:45

FWIW in shadow-cljs I have a :external js-provider which looks pretty similar to the new bundle stuff (ie. generate a .js file that other tools can process to load npm deps)

thheller11:04:54

I only did that as an experiment though so nobody uses it

thheller11:04:05

> the ecosystem effects I'm interested in

thheller11:04:08

what would that be?

dnolen11:04:54

you don't need CLJSJS like solutions to distribute ClojureScript libraries that depend on the JS ecosystem

dnolen11:04:09

that's the whole point of doing the bundle thing - not convenience

dnolen11:04:05

nor is it about tool X vs. X do CLJS builds - just don't care about that at all

dnolen11:04:42

the important thing is that any tool X can handle a CLJS dep that uses :npm-deps to rely on something

dnolen11:04:07

@thheller re: code-splitting, how do you handle requires that you can't process i.e. React components w/ image / CSS requires?

thheller11:04:44

just ignore them. they aren't too common in actual published libs (for the browser)

dnolen11:04:12

so users run another tool to handle them?

thheller11:04:17

they are common in react-native but shadow-cljs doesn't process that at all

dnolen11:04:39

so you can't for example use shadow code splitting to partition assets based on what components load on a page?

thheller11:04:26

it can in theory but it doesn't no

thheller11:04:13

but most libraries actually do not have css requires so not a big deal

dnolen11:04:13

re: webpack yeah not interested in trying to combine w/ their code splitting - it can't work

dnolen11:04:49

however the module grouping algorithm doesn't work on source, it only works on dependencies - so I think it can probably be made to work pretty easily

dnolen11:04:08

while processing node stuff through Closure is a hit or miss - our ability to compute the dep graph is pretty good

dnolen11:04:36

so we can just add these to the module grouping algorithm and mark them as coming from elsewhere

dnolen11:04:42

then webpack is just used to prepend that stuff

dnolen11:04:46

just as in the single file case

thheller11:04:27

for the :external case in shadow-cljs I intended that file to be loaded completely separately which effectively is sort of code-splitting out the npm code and the cljs code

thheller11:04:02

can't actually split npm deps but its usually good enough if you can separate out the npm parts and have that cacheable

dnolen11:04:26

yeah that's probably what we'll ship w/ now

dnolen11:04:35

and add the code splitting part as a separate step

dnolen13:04:06

it would be nice if all Clojure dep tools could fetch npm deps rather than ClojureScript caring about it, but I don't see that happening quickly in a synchronized fashion, @alexmiller thoughts?

dnolen13:04:10

unlike git deps it's not meaningful if only tools-deps can do it - every tool needs to be able to do it a la Maven

thheller13:04:42

if you intend to use webpack you'll have npm so just using npm seems fine to me

thheller13:04:55

I played with the idea of replacing it myself but its just not worth

thheller13:04:28

there are already alternatives like yarn anyways

dnolen13:04:35

@thheller of course it has to bottom out in npm

dnolen13:04:00

I'm just pointing there's no a universal Clojure way to have a node dep - curious if Alex has thoughts is all

dnolen13:04:32

ClojureScript being able to handle is a acceptable workaround for now - just looking at the much longer picture too

Alex Miller (Clojure team)13:04:18

I have actually thought about this a bit

Alex Miller (Clojure team)13:04:40

tools.deps is designed to procure and expand transitive deps through an abstraction (a few multimethods) and npm has everything needed to fulfill that

dnolen13:04:17

@alexmiller yeah that's why I was asking, you mentioned it before

Alex Miller (Clojure team)13:04:19

The question then is - what do you do with that? How does that relate to the classpath?

dnolen13:04:30

it doesn't need to relate to classpath

dnolen13:04:45

all that stuff just ends up in node_modules

dnolen13:04:49

something else will deal with it

dnolen13:04:42

my question is really more about whether you thought other tooling

dnolen13:04:51

since if it works w/ tool-deps not much of a win

dnolen13:04:04

not useful for Lein / Boot etc. w/o a lot more steps

Alex Miller (Clojure team)13:04:29

So how would you get at this behavior though? Programmatically by calling into tools.deps or do you want it to work via clj?

dnolen13:04:05

my expectation would be that just installs that stuff

dnolen13:04:39

to clarify we don't need classpath because node is just relative requires starting at node_modules

dnolen13:04:51

any build tool that is going to use that stuff understands that already

Alex Miller (Clojure team)13:04:08

Only in the project or in a shared place like m2?

dnolen13:04:39

I don't think we want to go there

dnolen13:04:52

yarn and npm have slightly different semantics

dnolen13:04:06

so local only

Alex Miller (Clojure team)13:04:08

I’m not suggesting anything, I just don’t know how any of this stuff works :)

Alex Miller (Clojure team)13:04:50

This is a major deviation from a model Rich and I have worked pretty hard to nail down so would need some real thought about whether this is the best place to do this (I’m a little worried to back into it by “this things looks kinda like that thing”)

Alex Miller (Clojure team)13:04:53

We have also been working on this tools.build stuff for artifact building and that may be a better fit, but we are still trying to figure out the shape of it

dnolen13:04:08

right, ok, just wanted to your thoughts

dnolen13:04:20

there's no issue IMO w/ us handling it - but wanted to get your initial thoughts before we release

dnolen16:04:44

@alexmiller are site builds automatic again?

dnolen17:04:22

@mfikes hrm, I guess because of spec.alpha and reader stuff we actually need Clojure 1.9? Or am I misremembering?

dnolen17:04:38

if that's true I'm not sure why we didn't bump this in the pom.template.xml?

mfikes17:04:20

Hmm... I can't recall

dnolen17:04:17

trying with Clojure 1.8

dnolen17:04:09

Clojure 1.8 works fine

dnolen18:04:23

@alexmiller kick it off whenever you have a chance

dnolen18:04:56

I removed a lot of stuff and modernized everything - clj and cljs.main only in the guides

dnolen18:04:05

@mfikes no rush but if you want to put together release notes that would be cool - I'm going to work on a post about :target bundle + updated Webpack guide

🎉 12
👍 4
dnolen19:04:30

if people want to try it out

dnolen19:04:51

for the ClojureScript dependency you'll either want to ./script/build and note the version that gets install locally

dnolen19:04:15

or git deps

dnolen19:04:16

{:deps {org.clojure/clojurescript {:git/url "" :sha "7792adf10e8961cbd5b0267a30a7d9655c770086"}}}

dpsutton19:04:08

@dnolen 1. npm --save-dev webpack webpack-cli needs install 2. same for the npm --save react react-dom (isn't --save implied by install?) 3. the clj -m cljs.main -co build.edn -v -c -s assumes a deps.edn file has been created already but that's not in the steps 4. going through these steps i got

Execution error (ExceptionInfo) at cljs.closure/build (closure.clj:3192).
:bundle-cmd :none failed
5. the stdout from the tempfile associated with this error was
:std-out
  18   │      "Hash: 980d631d7dd0f2217840\nVersion: webpack 4.42.1\nTime: 24ms\nBuilt at: 04/12/2020 2:56:09 PM\n\nERROR in Entry m
       │ odule not found: Error: Can't resolve './out/index.js' in '/Users/dan/projects/clojure/scratch/hello-bundler'\n"}
...
   :cause ":bundle-cmd :none failed",
  47   │   :data
  48   │   {:cmd ["npx" "webpack" "--mode=development"],
  49   │    :exit-code 2,
  50   │    :std-out

dnolen19:04:26

@dpsutton yeah I'm going through it now - updated

dpsutton20:04:24

in Firefox i'm getting > SyntaxError: import declarations may only appear at top level of a module and no console logging

dnolen20:04:10

@dpsutton did you try Chrome/Safari?

dpsutton20:04:55

Safari reported invalid { This machine is only two days old and doesn’t have chrome yet

dpsutton20:04:10

> import {npmDeps} from "./npm_deps.js"; seems to be the offending line. Is this the correct output of webpack?

dnolen20:04:48

@dpsutton I think you missed a step

dnolen20:04:24

I just copied and pasted everything and it worked (except the final step, advanced compilation + foreign override)

dnolen20:04:24

your issue not browser related - but something else - make sure you copy and paste unless you really understand the steps, no tweaks (happy to explain)

dpsutton20:04:52

Yeah tracing through to see which one I may have overlooked

dnolen20:04:06

:output-to must be out/index.js

dnolen20:04:27

entry: in webpack config must be out/index.js

dnolen20:04:02

and the final file must be out/main.js to see it in the default webserver

dnolen20:04:14

all of this can of course be changed - but these things must align

dnolen20:04:22

@dpsutton you can also drop :build-cmd and run webpack directly - that part is pure convenience

dpsutton20:04:42

ok i think i had missed installing webpack and webpack-cli when making my list earlier. sorry about that

dpsutton20:04:15

now everything is working in the browser. however, in the terminal i'm seeing

Exception in thread "Thread-386" java.lang.NullPointerException
	at clojure.core$deref_future.invokeStatic(core.clj:2300)
	at clojure.core$deref.invokeStatic(core.clj:2320)
	at clojure.core$deref.invoke(core.clj:2306)
	at cljs.repl.browser$repl_client_js.invokeStatic(browser.clj:116)
	at cljs.repl.browser$repl_client_js.invoke(browser.clj:115)
	at cljs.repl.browser$send_repl_client_page.invokeStatic(browser.clj:123)
	at cljs.repl.browser$send_repl_client_page.invoke(browser.clj:118)
	at cljs.repl.server$dispatch_request.invokeStatic(server.clj:192)
	at cljs.repl.server$dispatch_request.invoke(server.clj:182)
	at cljs.repl.server$handle_connection.invokeStatic(server.clj:206)
	at cljs.repl.server$handle_connection.invoke(server.clj:202)
	at cljs.repl.server$server_loop$fn__9753.invoke(server.clj:216)
	at clojure.core$binding_conveyor_fn$fn__5754.invoke(core.clj:2030)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.base/java.lang.Thread.run(Thread.java:830)
serves content just fine when i hit refresh in browser though

dnolen20:04:53

@dpsutton yeah that's side detail, will look into that later

👍 4
dnolen20:04:11

glad it's working!

dpsutton20:04:15

this is awesome. thanks so much!

dnolen20:04:35

there's definitely an issue w/ inferred externs, which I need to dig into

dnolen20:04:53

making sure that Reagent can work is good test case

dnolen20:04:59

@dpsutton yeah it's pretty cool, to be honest we could have done it a long time ago, now that I really see it, a variant of this had been suggested a few times by folks like @thheller - like just emitting require for CLJS namespaces so you can run JS tools

dnolen20:04:34

one problem is JS tools can't really handle CLJS well so the impact seemed minor

dnolen20:04:18

other issues are how to make this work for REPLs w/o being a pain in the butt - re-natal actually gave me the idea here - perhaps shadow does something similar don't know

dnolen20:04:35

anyway I think this is a pretty big win

dnolen20:04:45

since what this gives you is dead simple

dnolen20:04:55

we just generate stuff that you can pass onto your favorite JS tool

juhoteperi20:04:14

I will add this bundle target with webpack as a test environment to Reagent repository

juhoteperi20:04:08

I'll also remove cljsjs dependency in the next version, and update documentation to instruct how to provide React.

dnolen20:04:27

@juhoteperi cool! I do need to sort out the inferred externs issue first - somehow COMPILED is sneaking in there when trying this w/ Reagent

juhoteperi21:04:27

Looks like bundle-cmd isn't run for me automatically, but it works when I run webpack myself.

dnolen21:04:34

@juhoteperi it should work, it might have failed though it's strange there's no error for you

dnolen21:04:47

(though that could be a bug)

dnolen21:04:17

could be consolidated too

juhoteperi22:04:45

Couldn't find out reason. No error, no output and no file being written. Got the test suite running now, I passed the bundle to Karma with Webpack plugin, just needed a small workaround to get Karma to wait untill JS files loaded by Cljs are ready: https://github.com/reagent-project/reagent/pull/486/files

rakyi22:04:19

@juhoteperi do you have --ignore-scripts set for npm by chance? it’s npm option, but can be stored also in global config

dnolen23:04:51

ok I found the problem with advanced compilation - reagent project with react swapped in via webpack works fine under advanced compilation

dnolen23:04:58

@juhoteperi you should be able to reproduce on your REPL on your machine

dnolen23:04:31

(require '[clojure.java.shell :as sh]) (sh/apply ...) whatever your bundle command vector is

dnolen23:04:24

fwiw, I did get a case where the cmd failed for me and got an expected exception from the bad exit code