Fork me on GitHub
#shadow-cljs
<
2017-09-22
>
thheller08:09:45

hmm not sure where they are going with that, seems rather lumo specific

thheller08:09:05

and it doesn’t solve anything really

thheller08:09:29

the problem is that we have to deal with 2 package managers

thheller08:09:41

currently it is one for JS then one for CLJS

thheller08:09:57

when publishing to npm the problem gets worse because we can find JS and CLJS there

thheller08:09:06

but still need the usual maven style for CLJS

thheller08:09:21

unless everyone agrees to publish to npm which is unlikely at this point

Jon08:09:58

I'm wondering if Lumo made it, shadow-cljs might need to support it as well...

thheller08:09:43

I’ll implement it if CLJS core does

thheller08:09:26

in any case we should agree on a standard first, just dumping stuff whereever just isn’t going to work

thheller08:09:44

we probably should have a "cljs" property in package.json

thheller08:09:11

but I don’t have the brain capacity to think about this now, too busy trying to work out the JS dependency stuff

thheller08:09:23

browserify is making me angry .. just like webpack

Jon08:09:24

...as a standard in package.json

thheller08:09:37

in fact all of js is making be so angry, why can’t they all agree on one way to do things …

thheller08:09:54

currently trying to make jquery+bootstrap work

thheller08:09:08

but bootstrap doesn’t declare any dependency on jquery .. it just expects it as a global

Jon08:09:11

because js is not controlled by one single company

thheller08:09:40

which in turn can only be configured by package.json

thheller08:09:58

as if I had a package.json for every build lying arround

Jon08:09:21

I think shadow-cljs does not have to support all things in js world. things like Bootstrap may be troublesome too even in Webpack.

thheller08:09:57

yeah but its one of those packages ppl expect to use

thheller08:09:16

jquery works, react works, most packages around react work

thheller08:09:20

bootstrap doesn’t

Jon08:09:09

well, then it's Bootstrap's problem

thheller08:09:00

hmm you might be right

Jon08:09:08

sometimes we still use CDN for some of the libraries, even we have Webpack nowadays

thheller08:09:23

yeah but for that I need browserify-shim as well

thheller08:09:27

the shim part is not the issue

thheller08:09:35

that it can only be configured by package.json is

thheller08:09:46

I should just fork it I guess

Jon08:09:28

fork browserify-shim?

thheller08:09:08

not like its getting many updates these days

Jon08:09:15

in the website I previously worked for view-source:https://h5.ele.me/msite/ lots of can be found

thheller08:09:25

yeah thats stuff I need to support anyways

Jon08:09:26

somehow sounds like you are creating your own bundler for CommonJS

thheller08:09:37

ie. not using React from npm but from global in page via cdn

Jon09:09:04

well, React 😅

thheller09:09:14

just using react as an example

thheller09:09:18

ANYTHING from a cdn

thheller09:09:23

jquery, react, whatever

Jon09:09:38

React is following CommonJS, Bootstrap isn't

thheller09:09:06

you misunderstand me

thheller09:09:22

it doesn’t matter what the packages do

Jon09:09:35

supporting libraries from CDN?

thheller09:09:35

some projects will still have react via CDN in their page

Jon09:09:43

hh, right..

thheller09:09:46

so I must use that and NOT the package from npm

thheller09:09:54

otherwise it would load react twice

Jon09:09:31

yeah, there's a feature in Webpack doing that, many another loader..

thheller09:09:00

yeah .. shims … that is what I’m talking about with browserify-shim

thheller09:09:18

the solution is there … it just isn’t configurable the way I’d like

thheller09:09:28

I can’t put config in package.json

Jon09:09:36

it's so complicated, I always hope bundlers like Webpack can do it for me

thheller09:09:40

well I can but I’d really like to avoid that 😛

thheller09:09:22

ProvidePlugin is basically equal to browserify-shim

Jon09:09:25

package.json vs. shadow-cljs.edn?

thheller09:09:50

forget about it, not important

Jon09:09:19

🙂 hard for me to understand all of that anyway

thheller09:09:42

its impossible to understand

thheller09:09:55

every package does its own quirky little thing

thheller09:09:06

webpack is the worst offender of them all to be honest

Jon09:09:10

yeah... js things always annoying. actually it has been a lot better since people using CommonJS and Webpack

thheller09:09:17

instead of trying to find a “standard” way to do things

thheller09:09:24

it has workarounds for just about everything

thheller09:09:32

thats why configuring it is so painful

Jon09:09:57

"wort offender" -> "worst offender"?

Jon09:09:18

I guess that's one of the reasons js developers not picking ClojureScript... js world is so crazy, cljs is not prepared for it

thheller09:09:26

I don’t understand why some of the most popular packages still do their own shit

thheller09:09:40

at least they should have found some kind of agreement how to do things

Jon09:09:09

I don't get it, are you referring to npm?

thheller09:09:18

bootstrap in particular

Jon09:09:44

maybe it's for old websites?

thheller09:09:52

the bootstrap npm package requires custom config in EVERY build tool

thheller09:09:01

why even publish to npm …

Jon09:09:15

I heard some developers just use it for websites, while they are backend developers

thheller09:09:10

it has 115K stars on github, surely there is someone that could fix their packaging?

Jon09:09:03

sad fact 😥

Jon09:09:43

I remembered that doing bundling for multi-pages websites is still difficult even with Webpack, plus new tech like Service Workers or somethings.

thheller09:09:46

that code is the problem … so easy to fix

thheller09:09:19

wait what the actual f*

thheller09:09:15

I give up … need to do other stuff or I’ll loose my sanity

Jon09:09:34

what's happening now 😓

thheller09:09:26

can’t look at bootstrap anymore … it just won’t be supported for now

Jon09:09:49

maybe relying on Webpack for it is the best option

thheller09:09:08

@jiyinyiyong the issue is not which build tool I use

thheller09:09:20

the issue is that EVERY build tool requires special config just for bootstrap

thheller09:09:04

well the good news is that many other packages just work

thheller09:09:24

might push a preview release today so you can start testing 😉

Jon09:09:48

sounds nice!

Jon09:09:59

somehow most of my projects are using cljs code only due to the bad support for npm packages before 😂

thheller09:09:24

that is not a bad thing, CLJS code is better than JS 😉

Jon09:09:05

....trying to find projects that using npm packages..

thheller10:09:49

if anyone wants to try it I just published shadow-cljs@next

thheller10:09:22

will finish the blog post now

thheller10:09:33

it will not compile anything that uses cljsjs packages … but I need some testers 😛

thheller11:09:12

@bolasblack the shadow-cljs@next release I just pushed should fully support :cache-root. It will probably break everything else though 😉

Jon12:09:45

=>> yarn watch
yarn watch v0.24.6
$ shadow-cljs watch app
shadow-cljs - config: /Users/chen/repo/minimal-xyz/minimal-shadow-cljs-browser/shadow-cljs.edn version: SNAPSHOT
shadow-cljs - updating dependencies
shadow-cljs - dependency update failed - Could not find artifact thheller:shadow-cljs:jar:SNAPSHOT in central ()
error Command failed with exit code 1.

Jon12:09:52

is SNAPSHOT the correct version name?

thheller12:09:21

no, i broke closure-defines

Jon12:09:26

Retrieving...

Jon12:09:46

it's still retrieving >o<

thheller12:09:06

retrieving what?

Jon12:09:23

yarn watch v0.24.6
$ shadow-cljs watch app
shadow-cljs - config: /Users/chen/repo/minimal-xyz/minimal-shadow-cljs-browser/shadow-cljs.edn version: 2.0.0-alpha3
shadow-cljs - updating dependencies
Retrieving thheller/shadow-cljs/2.0.0-alpha3/shadow-cljs-2.0.0-alpha3.pom from 
Retrieving org/clojure/clojurescript/1.9.908/clojurescript-1.9.908.pom from 
Retrieving org/clojure/google-closure-library/0.0-20170809-b9c14c6b/google-closure-library-0.0-20170809-b9c14c6b.pom from 
Retrieving org/clojure/google-closure-library-third-party/0.0-20170809-b9c14c6b/google-closure-library-third-party-0.0-20170809-b9c14c6b.pom from 
Retrieving org/clojure/tools.reader/1.0.5/tools.reader-1.0.5.pom from 
Retrieving com/google/javascript/closure-compiler-unshaded/v20170910/closure-compiler-unshaded-v20170910.pom from 
Retrieving com/google/javascript/closure-compiler-main/v20170910/closure-compiler-main-v20170910.pom from 
Retrieving com/google/javascript/closure-compiler-parent/v20170910/closure-compiler-parent-v20170910.pom from 
Retrieving com/google/javascript/closure-compiler-externs/v20170910/closure-compiler-externs-v20170910.pom from 
Retrieving org/clojure/google-closure-library/0.0-20170809-b9c14c6b/google-closure-library-0.0-20170809-b9c14c6b.jar from 
Retrieving org/clojure/tools.reader/1.0.5/tools.reader-1.0.5.jar from 
Retrieving org/clojure/clojurescript/1.9.908/clojurescript-1.9.908.jar from 
Retrieving org/clojure/google-closure-library-third-party/0.0-20170809-b9c14c6b/google-closure-library-third-party-0.0-20170809-b9c14c6b.jar from 
Retrieving com/google/javascript/closure-compiler-unshaded/v20170910/closure-compiler-unshaded-v20170910.jar from 
Retrieving com/google/javascript/closure-compiler-externs/v20170910/closure-compiler-externs-v20170910.jar from 

thheller12:09:36

should be fine to just kill the process and try again. maybe the connection died or something

Jon12:09:38

and poor network from China mainland

thheller12:09:55

then just wait 😉

thheller12:09:35

great just found the next bug 😉

thheller12:09:28

you might get a Caused by: java.lang.IllegalArgumentException: No matching field found: close for class sun.net.www.protocol.jar.JarURLConnectio

thheller12:09:13

upgrade should fix it

Jon12:09:38

Retrieving thheller/shadow-cljs/2.0.0-alpha3/shadow-cljs-2.0.0-alpha3.jar from 
shadow-cljs - dependencies updated
shadow-cljs - re-building aot cache on startup, that will take some time.
shadow-cljs - starting ...
Exception in thread "main" java.lang.IllegalArgumentException: No matching field found: close for class sun.net.www.protocol.jar.JarURLConnection, compiling:(compiler.clj:32:3)
	at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3676)
	at clojure.lang.Compiler$DefExpr.eval(Compiler.java:457)
	at clojure.lang.Compiler.compile1(Compiler.java:7552)
	at clojure.lang.Compiler.compile(Compiler.java:7619)
	at clojure.lang.RT.compile(RT.java:408)
	at clojure.lang.RT.load(RT.java:453)
	at clojure.lang.RT.load(RT.java:421)
	at clojure.core$load$fn__6368.invoke(core.clj:6008)
	at clojure.core$load.invokeStatic(core.clj:6007)
	at clojure.core$load.doInvoke(core.clj:5991)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5812)
	at clojure.core$load_one.invoke(core.clj:5807)
	at clojure.core$load_lib$fn__6313.invoke(core.clj:5852)
	at clojure.core$load_lib.invokeStatic(core.clj:5851)

thheller12:09:57

fixed in alpha4

Jon12:09:16

closure compiler is so huge

thheller12:09:20

is it? should be 7mb or so?

Jon12:09:23

shadow-cljs - starting ...
failed to start service
{:id :dev-http}
ExceptionInfo: failed to start service
	clojure.core/ex-info (core.clj:4725)
	clojure.core/ex-info (core.clj:4725)
	shadow.runtime.services/start-many/fn--14458 (services.clj:135)
	shadow.runtime.services/start-many (services.clj:126)
	shadow.runtime.services/start-many (services.clj:105)
	shadow.runtime.services/start-all (services.clj:144)
	shadow.runtime.services/start-all (services.clj:139)
	shadow.cljs.devtools.server/start-system (server.clj:150)
	shadow.cljs.devtools.server/start-system (server.clj:94)
	shadow.cljs.devtools.server/start! (server.clj:180)
	shadow.cljs.devtools.server/start! (server.clj:172)
	shadow.cljs.devtools.server/start! (server.clj:175)
	shadow.cljs.devtools.server/start! (server.clj:172)
	shadow.cljs.devtools.server/from-cli (server.clj:260)
	shadow.cljs.devtools.server/from-cli (server.clj:251)
	shadow.cljs.devtools.cli/main (cli.clj:92)
	shadow.cljs.devtools.cli/main (cli.clj:74)
	clojure.core/apply (core.clj:657)
	clojure.core/apply (core.clj:652)
	shadow.cljs.devtools.cli/-main (cli.clj:132)
	shadow.cljs.devtools.cli/-main (cli.clj:130)
	clojure.lang.Var.applyTo (Var.java:702)
	clojure.core/apply (core.clj:657)
	clojure.main/main-opt (main.clj:316)
	clojure.main/main-opt (main.clj:312)
	clojure.main/main (main.clj:423)
	clojure.main/main (main.clj:386)
	clojure.lang.Var.applyTo (Var.java:702)
	clojure.main.main (main.java:37)
Caused by:
invalid config
In: [:builds :npm 1] val: {:id :npm, :target :npm-module, :output-dir "node_modules/shadow-cljs"} fails spec: :shadow.build.config/build at: [:builds 1] predicate: (contains? % :build-id)

error Command failed with exit code 1.
=>>

Jon12:09:56

many it's poor network, 7M is not large

thheller12:09:13

is that in your config?

thheller12:09:32

oh right the built-in config was broken

Jon12:09:43

{:source-paths ["src"]
 :dependencies []
 :builds {:app {:output-dir "target/"
                :asset-path "."
                :target :browser
                :modules {:main {:entries [app.main]}}
                :devtools {:after-load app.main/reload!}}}}

Jon12:09:55

no :npm-module

thheller12:09:19

looks like you called shadow-cljs compile npm and not shadow-cljs compile app?

Jon12:09:44

should be "watch": "shadow-cljs watch app",

Jon12:09:10

strange, no npm found in my config or package.json

thheller12:09:18

yeah its the built-in config

thheller12:09:30

dammit I really need to test outside the project itself

thheller12:09:42

next version coming up 😉

Jon12:09:29

[:app] Configuring build.
[:app] Compiling ...
[:app] Build failure:
Failed to run: ./node_modules/.bin/browserify -d -g envify -r hsl
{:tag :shadow.build.targets.browser/browserify, :command ["./node_modules/.bin/browserify" "-d" "-g" "envify" "-r" "hsl"]}
ExceptionInfo: Failed to run: ./node_modules/.bin/browserify -d -g envify -r hsl
	clojure.core/ex-info (core.clj:4725)
	clojure.core/ex-info (core.clj:4725)
	shadow.build.targets.browser/bundle-js/fn--25574/fn--25583 (browser.clj:461)
	shadow.build.targets.browser/bundle-js/fn--25574 (browser.clj:449)

Jon12:09:46

Caused by:
IOException: Cannot run program "./node_modules/.bin/browserify": error=2, No such file or directory
	java.lang.ProcessBuilder.start (ProcessBuilder.java:1048)
	sun.reflect.NativeMethodAccessorImpl.invoke0 (NativeMethodAccessorImpl.java:-2)

Jon12:09:35

browserify not installed, do I have to install it by myself?

thheller12:09:09

hmm no shadow-cljs should depend on it

thheller12:09:37

ah hmm do you have shadow-cljs dependency in your project?

thheller12:09:40

or just the global?

Jon12:09:57

dependencies:
   { mkdirp: '^0.5.1',
     'readline-sync': '^1.4.7',
     'shadow-cljs-jar': '^0.10.0',
     'source-map-support': '^0.4.15',
     ws: '^3.0.0' },

Jon12:09:01

inside project

Jon12:09:18

did you put that in devDependencies?

thheller12:09:46

thats the old deps?

Jon12:09:06

I queried it from npm info shadow-cljs

thheller12:09:09

"dependencies": {
    "browserify": "^14.4.0",
    "envify": "^4.1.0",
    "mkdirp": "^0.5.1",
    "readline-sync": "^1.4.7",
    "shadow-cljs-jar": "^0.10.0",
    "source-map-support": "^0.4.15",
    "uglifyify": "^4.0.4",
    "ws": "^3.0.0"
  }

Jon12:09:20

strange.

Jon12:09:28

mine was from npm info shadow-cljs

thheller12:09:30

npm info shadow-cljs@next

thheller12:09:48

that @next thing is getting annoying already 🙂

thheller12:09:16

but you need to have shadow-cljs added to your project so browserify gets installed into node_modules and not the global one

Jon12:09:21

I installed with yarn

Jon12:09:49

the package browserify is installed in node_modules/, but I got no .bin/browserify

Jon12:09:58

trying to install locally first

thheller12:09:50

hmm looks like yarn doesn’t install bin entries from dependencies

Jon12:09:11

[:app] Build completed. (112 files, 1 compiled, 0 warnings, 9.15s)
Sep 22, 2017 8:57:29 PM clojure.tools.logging$eval486$fn__489 invoke
WARNING: build-update: [[:shadow.build.classpath/resource "app/main.cljs"]]
[:app] Compiling ...
[:app] Build completed. (112 files, 1 compiled, 0 warnings, 0.61s)
Sep 22, 2017 8:57:45 PM clojure.tools.logging$eval486$fn__489 invoke
WARNING: build-update: [[:shadow.build.classpath/resource "app/lib.cljs"]]
[:app] Compiling ...
[:app] Build completed. (112 files, 2 compiled, 0 warnings, 0.35s)
Sep 22, 2017 8:57:47 PM clojure.tools.logging$eval486$fn__489 invoke
WARNING: build-update: [[:shadow.build.classpath/resource "app/lib.cljs"]]

Jon12:09:28

and it does not call my reload! function this time

thheller12:09:54

hmm checking, might have messed that up

thheller13:09:49

I was so busy with testing the npm stuff I never checked live-reloading 😛

Jon13:09:33

do you have a todolist for bugs 😂

thheller13:09:29

I do, just on paper though

thheller13:09:08

hmm the yarn issue is marked as merged

thheller13:09:32

but I don’t get the proper symlink either

thheller13:09:52

ah lol it symlink into node_modules/shadow-cljs/node_modules/.bin/browserify

thheller13:09:08

guess I’ll change that

thheller13:09:52

I get npm does something else entirely .. this is gonna be fun

thheller13:09:16

yep npm links it directly into node_modules/.bin

thheller13:09:36

great … another annoying thing I have to work around

thheller13:09:09

[email protected] should fix the reload issue

Jon13:09:10

confirmed reload..

thheller13:09:28

hmm dammit reload won’t work when you add JS deps

thheller13:09:06

ah well can’t have everything

thheller13:09:45

@jiyinyiyong did it work though? you are the first tester 😉

Jon13:09:14

didn't try adding js deps

thheller13:09:46

nah I mean did it work at all?

Jon13:09:18

for reload after file change, yes

Jon13:09:27

for adding new npm package, no:

Jon13:09:30

[:app] Compiling ...
[:app] Build failure:
Could not find js dependency "shortid", either install the npm package or define in :js-options
{}
ExceptionInfo: Could not find js dependency "shortid", either install the npm package or define in :js-options
	clojure.core/ex-info (core.clj:4725)
	clojure.core/ex-info (core.clj:4725)
	shadow.build.resolve/resolve-string-require (resolve.clj:127)
	shadow.build.resolve/resolve-string-require (resolve.clj:110)
	shadow.build.resolve/resolve-require (resolve.clj:227)
	shadow.build.resolve/resolve-require (resolve.clj:220)
	shadow.build.resolve/resolve-deps/fn--19695 (resolve.clj:56)
	clojure.lang.PersistentVector.reduce (PersistentVector.java:341)

thheller13:09:58

did you npm install shortid? or yarn

Jon13:09:42

success Saved 2 new dependencies.
├─ [email protected]
└─ [email protected]
✨  Done in 5.28s.
=>> l node_modules/sh
sha.js/          shadow-cljs-jar/ shell-quote/
shadow-cljs/     shasum/          shortid/
=>> l node_modules/shortid/
CHANGELOG.md  README.md     lib/
LICENSE       index.js      package.json

Jon13:09:52

I did yarn add shortid

thheller13:09:25

hmm does it work after a restart?

Jon13:09:36

not even after restart

Jon13:09:04

=>> coffee
coffee> require 'shortid'
Expression assignment to _ now disabled.
{ [Function: generate]
  generate: [Circular],
  seed: [Function: seed],
  worker: [Function: worker],
  characters: [Function: characters],
  decode: [Function: decode],
  isValid: [Function: isShortId] }

Jon13:09:12

confirmed installed

thheller13:09:37

and thats in the directory the project is in?

Jon13:09:12

very strange

Jon13:09:45

'use strict';
module.exports = require('./lib/index');

Jon13:09:15

=>> yarn --version
0.24.6

Jon13:09:23

meanwhile my yarn is too old

thheller13:09:35

ah .. hmm dammit

thheller13:09:17

I had hoped to get not having to deal with this issue for a while 😉

Jon13:09:00

looks like I have to use [email protected]

thheller13:09:02

currently don’t support a map 😉

Jon13:09:39

browserify?

thheller14:09:02

nah the package.json uses a map for "browser", I only support strings at the moment

Jon14:09:30

didn't know that, what's it used for?

thheller14:09:42

to replace require calls

Jon14:09:07

well, unexpected

thheller14:09:09

another wonderful thing from the js world 😉

Jon14:09:37

😅 so I never realized Webpack handled that

thheller14:09:58

yeah the fun part is that a string is also allowed

thheller14:09:30

if you its a string its a replacement for "main", if its a map the functionality is completely different

thheller14:09:40

and instead redirects actual require calls

thheller14:09:00

guess I can’t ignore that 😞

Jon14:09:10

didn't see docs or specs

thheller14:09:55

oh there is actual “spec” for that

thheller14:09:04

guess it would have been to hard to use 2 separate keys …

Jon14:09:23

main and browser?

thheller14:09:38

browser and browser-overrides or so

thheller14:09:55

that browser has 2 different functions is the annoying part

Jon14:09:46

annoying...

thheller14:09:09

I’m always amazed that absolutely anything works in JS land

thheller14:09:36

the code has a require for var clusterWorkerId = require('./util/cluster-worker-id')

thheller14:09:58

but maps "./lib/util/cluster-worker-id.js": "./lib/util/cluster-worker-id-browser.js"

thheller14:09:57

so I have to first check is .js exists since .json is also allowed

thheller14:09:03

then check if there is an override

thheller14:09:15

totally fun working through all this shit

Jon14:09:04

😂 even amazing people making Webpack, Browserify, RequireJS, SeaJS for run

thheller14:09:05

I’m close to writing one myself .. they all just don’t do exactly what I need

thheller14:09:35

but dealing with all those crazy edge cases would probably drive me crazy

thheller14:09:39

even more than it does now 😉

Jon14:09:15

so it would be smart if a language just compiles to CommonJS/ES6 module...

thheller14:09:33

it would be a whole lot simpler if everything used ES6

thheller14:09:01

but that won’t happen anytime soon

thheller14:09:41

and in the meantime there will probably be a billion other things people invent to do package management

Jon14:09:04

=>> yarn
yarn install v1.0.2
[1/4] 🔍  Resolving packages...
warning Lockfile has incorrect entry for "[email protected]". Ignoring it.
Couldn't find any versions for "shadow-cljs" that matches "2.x"
? Please choose a version of "shadow-cljs" from this list: 2.0.0-alpha6
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 6.20s.
=>> l node_modules/.bin/
JSONStream             browserify             envify                 he                     insert-module-globals  mkdirp                 sha.js                 umd

Jon14:09:29

browserify command installed now

thheller15:09:59

@jiyinyiyong did you try other npm packages or just shortid?

Jon15:09:25

just hsl and shortid

thheller15:09:24

got shortid working

thheller15:09:17

oops that contains a rogue prn statement 😉

thheller15:09:39

well need to fix something else anyway

Jon15:09:16

confirmed~