Fork me on GitHub
#shadow-cljs
<
2018-07-19
>
thheller05:07:54

@kenny no. but you can call clj -A:xyz -m shadow.cljs.devtools.cli compile foo instead of shadow-cljs compile foo

thheller05:07:23

but consider this https://code.thheller.com/blog/shadow-cljs/2018/02/08/problem-solved-source-paths.html. it is generally not useful or required to separate source paths for CLJS.

Yehonathan Sharvit06:07:58

@thheller I want to write a cljs lib that wraps an npm module (in my case js-yaml). What should be the :target for this cljs lib? Should it be :npm-module?

thheller06:07:50

@viebel nothing. CLJS libs are uncompiled source files packaged in a jar. meaning they aren't configured at all for shadow-cljs since it has nothing to build

Yehonathan Sharvit06:07:11

what about the package.json file of the lib? I will include it the jar. right?

thheller06:07:43

you create a deps.cljs at the root with {:npm-deps {"js-yaml" "the-version"}}

thheller06:07:56

package.json won't be used at all

thheller06:07:05

since npm doesn't look into .jar files

Yehonathan Sharvit07:07:45

And then in an app that consumes my cljs-yaml lib I won't need to manually install js-yaml. Right?

thheller07:07:23

if you are consuming with shadow-cljs then js-yaml will be installed automatically if deps.cljs specifies it

thheller07:07:34

for standard CLJS it might be more tricky and I honestly can't tell you how it works there

Yehonathan Sharvit07:07:12

It's ok for me to consume with shadow-cljs

Yehonathan Sharvit07:07:26

But it doesn't work either - for the moment

Yehonathan Sharvit07:07:53

āžœ  shadow-cljs-app git:(master) āœ— shadow-cljs compile browser
shadow-cljs - config: /mnt/c/Users/USER/prj/shadow-cljs-app/shadow-cljs.edn  cli version: 2.4.22  node: v8.10.0
shadow-cljs - updating dependencies
shadow-cljs - dependencies updated
shadow-cljs - starting ...
[:browser] Compiling ...
The required JS dependency "js-yaml" is not available, it was required by "cljs_yaml/core.cljs".

Searched in:/mnt/c/Users/USER/prj/shadow-cljs-app/node_modules

You probably need to run:
  npm install js-yaml

See: 

thheller07:07:34

what are the contents of the cljs-yaml.jar?

Yehonathan Sharvit07:07:22

it doesn't contain deps.cljs

Yehonathan Sharvit07:07:35

I published it with lein deploy clojars

thheller07:07:06

where do you have the deps.cljs?

Yehonathan Sharvit07:07:17

in the root folder

thheller07:07:39

needs to be in one source path

Yehonathan Sharvit07:07:39

@thheller shadow-cljs is adding js-yaml automatically to the package.json of my app

thheller07:07:51

yes it must

Yehonathan Sharvit07:07:09

I am not sure this is the best option

thheller07:07:16

its the only option

thheller07:07:29

npm can remove things when not listed in package.json

thheller07:07:47

or rather yarn does that by default. if you remove something from package.json it is uninstalled

Yehonathan Sharvit07:07:19

What I can do in my app is never touch the dependencies in package.json by myself and only works with deps.cljs. And let shadow-cljs manage the dependencies in package.json

thheller07:07:55

you should really be using package.json + npm as its meant to be by npm

thheller07:07:06

using shadow-cljs to "emulate" it will just suck

thheller07:07:30

you are going to run into conflicts eventually and gonna need npm to resolve them anyways

thheller07:07:54

I hate npm too but thats the way it works in the JS world. can't do anything to change that.

Yehonathan Sharvit07:07:59

I don't understand. What is the best place to declare my npm dependencies? package.json or deps.cljs?

thheller07:07:29

package.json in your project. deps.cljs for libs

Hukka07:07:01

Hm, I understood that @viebel is trying to make a lib for all the npm deps, and then just mark them as clj deps, letting the build tool then download the subnpmdeps. But that's discouraged?

Hukka07:07:33

I suppose that's the cljsjs-way, but without doing externs etc

Hukka07:07:52

@thheller Does shadow have some kind of concise document on how it works, internally? I often have the feeling that I need to know how the tools I use to not get cut in edge cases. For instance, I was wondering why my .-js-property accesses didn't get mangled by advanced closure, when using plain npm requires. The libraries themselves are not build with advanced, so the names stay correct on that end, but my cljs code should be.

thheller07:07:04

@tomi.hukkalainen_slac since shadow-cljs processes all the JS it also extracts all properties found in those JS files and uses those for externs inference

thheller07:07:58

also when using (:require ["thing" :as t]) all accesses to t/foo are automatically added to externs

thheller08:07:25

this is probably the most concise document about how it works currently https://code.thheller.com/blog/shadow-cljs/2018/06/15/why-not-webpack.html

Hukka08:07:04

In this case it was callback code, again. So I'm making a function that I give to a JS thingy. Then the thingy calls my function, giving it an #js object as parameter. Then I do property access on that object

Hukka08:07:58

Before, on vanilla cljs (though not on the newest version) I tried to make externs for that, but ended up with metadata helpers everywhere, and eventually just used oops

Hukka08:07:18

But with shadow, the plain .- and even set! just worked

Hukka08:07:04

Though perhaps that's just the new and improved externs inference working better... which means I don't know how the plain cljs works either (and certainly I don't)

thheller08:07:34

shadow-cljs does a lot more externs inference since it processes all the JS

thheller08:07:02

for example when it finds exports.foo = ... it will add foo to externs just in case you call it

Hukka08:07:22

It uses the child function / render prop paradigm, so everything you do with it is always making a function that downshift will then call when it needs to, and then you work on the parameter object

Hukka08:07:28

It's not a full example (missing the reagent atom setup and reset function), but shows how to change the input element functionality inside the downshift autocomplete component.

Hukka08:07:46

The FIXME is outdated, since it still works in shadow-cljs release app šŸ™‚

thheller08:07:03

safest thing to do would be (fn [^js downshiftProps] ...)

thheller08:07:22

then it should probably work in normal CLJS as well

thheller08:07:40

(.-nativeEvent.-preventDownshiftDefault event) thats not valid though

thheller08:07:01

(.. event -nativeEvent -preventDownshiftDefault)

Hukka08:07:53

Odd, didn't get any errors or warnings

thheller08:07:17

yeah since JS allows setting any arbitrary props. it just won't have any effect šŸ˜‰

Hukka08:07:35

Is it setting a property named 'nativeEvent.-preventDownshiftDefault'?

thheller08:07:08

probably "_preventDownshiftDefault"

thheller08:07:28

not sure actually. maybe it accidentally works

Hukka08:07:25

That line might not be even necessary, then. The method call matters the most.

Hukka08:07:49

Anyway, I'm miffed when things don't work as I expected, even if the work unexpectedly šŸ˜•

Hukka08:07:09

I could just pepper ^js in every callback like that, but it feels like cargo culting

thheller08:07:06

set :compiler-options {:infer-externs :auto} in your build config

thheller08:07:14

and set it whenever you get a warning

thheller08:07:20

otherwise you should be fine

Hukka08:07:20

Could :compiler-options {:infer-externs :all} help...

Hukka08:07:23

Ah, :auto instead

thheller08:07:40

I don't recommend :all since that will complain about files you don't control

Hukka08:07:40

infer-externs doesn't give any warnings. I kinda would like to try if the same works on new clojurescript, but I'm guessing I couldn't get the deps installed šŸ˜…

Hukka08:07:36

I now have a fully working minimal set with lein, ring/jetty, uberjars etc. Time to move the real project into shadow

Hukka08:07:19

šŸ˜” maybe lunch first

thheller08:07:20

probably best to keep the CLJ part with lein and stuff

thheller08:07:31

shadow-cljs is focused and built on CLJS

thheller08:07:55

while you can run CLJ normally through it it will never be optimized for it

thheller08:07:58

ie. no uberjars

claudiu09:07:21

not really sure to be honest. šŸ™‚ I am deploying my test-app to google-appengine, and that's the format there.

claudiu09:07:22

Added lein-tools-deps to my project, seems to work for development but the uberwar seems to compile multiple times the same file, and when I run it I get some strange errors.

dominicm09:07:36

I can try find some time to look into it for you.

dominicm09:07:53

Looks like a jar with a specific fs layout, shouldn't be too difficult to get working with pack, I'm a little worried about git dependencies, that might need some testing.

claudiu10:07:11

Cool šŸ™‚ Could help with testing git & normal, and see how it matches the lein one.

claudiu11:07:26

Having the uberwar option would be really awesome.

dominicm17:07:18

Is there an easy way to test an uberwar locally?

claudiu09:07:41

I have only used uberwar in appengine context. The simplest way to test the war I can think of is : https://gist.github.com/claudiuapetrei/540c8aca1ef53d86f75f123b319a3703

dominicm10:07:01

Thanks, I'll take a look

Hukka08:07:41

I mean lein makes shadow compile the cljs into resources, and then bundles the jetty server that servers that along with the dynamic api code

Hukka08:07:18

I was thinking if I should make this into a tutorial repository, but not sure if there's demand/use for it.

Hukka08:07:48

Building the thing commit by commit, where you could follow the workflow, and document the cli commands into the readme

thheller08:07:07

sounds like a good idea. I still need to document the whole "migrating from X ..." parts but since I have never used X thats pretty tough

plamen09:07:37

@thheller A lot of thanks for the hint from yesterday about the reload/history initialisation!

Hukka09:07:23

@thheller Coming to clojutre? We'd get you a beer (or whatever floats your boat ;)

Hukka10:07:52

Hm. If I make a clojurescript library and use the :require :default syntax, will vanilla cljs users have problems importing my lib?

p-himik11:07:41

Is there any particular reasons for using lists in the table at https://shadow-cljs.github.io/docs/UsersGuide.html#_using_npm_packages? I had to scratch my head for a minute since before that page I've seen lists in :refer only in really old code - now all examples use vectors.

colindresj12:07:38

Clojure doesnā€™t care if you use a list or a vector, itā€™s just become idiomatic to use a vector. I donā€™t want to speak for him, but I suspect that @thheller might be using a list to help show that itā€™s a JS dependency being brought in (same reasoning behind strings vs symbols in require)

Hukka12:07:38

On my real project I'm getting a bunch of

Jul 19, 2018 3:32:07 PM shadow.build.classpath invoke
WARNING: provide conflict for #{cljs_time.format cljs-time.format} provided by cljs_test/generated_test_suite.out/cljs_time/format.js and {"/cljs_time/format.cljs" #{cljs-time.format}}
Jul 19, 2018 3:32:07 PM shadow.build.classpath invoke
WARNING: provide conflict for #{cljs_time.predicates cljs-time.predicates} provided by cljs_test/generated_test_suite.out/cljs_time/predicates.js and {"/cljs_time/predicates.cljs" #{cljs-time.predicates}}
Jul 19, 2018 3:32:07 PM shadow.build.classpath invoke
INFO: filename violation for ns cljs.nodejs, got: cljs_test/generated_test_suite.out/cljs/nodejs.cljs expected: cljs/nodejs.cljs (or .cljc)
I've no idea what that generated_test_suite is, and I'm not using cljs-test myself. I tried googling and seems to point at boot-cljs-test... which is really odd. Any idea what's going on?

Hukka12:07:27

cljs-time seems to use that boot-cljs-test, but I don't see how it could be running

Hukka12:07:40

Hm. The compiled js dir has cljs_test.generated_test_suite.out.cljs.core.constants.js and cljs_test.generated_test_suite.out.cljs_time.internal.core.js

Hukka13:07:54

Well, got that fixed by upgrading the cljs-time package. New one doesn't have the conflicting files. But I wonder why lein-cljsbuild never complained about that

mitchelkuijpers14:07:37

We just upgraded from shadow-cljs 2.4.7 to 2.4.22 and now it breaks on release with: ["scroll-into-view-if-needed" :default scroll-into-view-if-needed] it creates P4a["default"] but P4a contains:

ʒ (a,b){if(b===Object(b)&&0!==Object.keys(b).length&&
"function"===typeof b.behavior)return b.behavior((0,e.default)(a,b));b=!1===b?{block:"end",inline:"nearest"}:b===Object(b)&&0!==Object.keys(b).lengā€¦

thheller16:07:45

@mitchelkuijpers odd. did you upgrade whatever npm package that is as well? maybe they changed the export?

mitchelkuijpers07:07:25

The problem was that our build server had npm install --no-package-lock but that did not only not create a new package-lock.json but it would also ignore it... Now we updated node and changed the command to npm ci and life is great again ^^

thheller16:07:27

@p-himik I always use lists for :refer. personal preference I guess. CLJ(S) don't care, both are equally valid

thheller16:07:00

@tomi.hukkalainen_slac that looks like some generated code is on the classpath, try shadow-cljs clj-repl and then ( "cljs_test/generated_test_suite.out/cljs_time/format.js")

thheller16:07:52

that should tell you where its coming from

Hukka18:07:03

@thheller It was, yes, and was fixed in a new release of the lib. But I was surprised that the old cljsbuild based system didn't tell me about that

Hukka18:07:46

Is shadow using the clojurescript compiler, or is it completely separate and the common point is the closure compiler?

thheller19:07:31

@tomi.hukkalainen_slac shadow-cljs indexes the classpath and is a bit overeager in some cases. its is the standard clojurescript compiler but the way things are resolved is a bit different, to make the custom JS resolvers possible etc.

Hukka19:07:03

Would be great to read a guide that opens the hood and shows different parts of the engine

Hukka19:07:26

But no doubt there's plenty of other things to do too...

thheller19:07:52

yeah it takes forever to write and I never know where to start. I tried starting a blog series about the differences https://code.thheller.com/blog/shadow-cljs/2018/02/08/problem-solved-source-paths.html

thheller19:07:16

will probably continue that when I have some more time

Hukka19:07:18

Does anyone else know the innards well enough, and could be prodded?

thheller19:07:18

of shadow-cljs? probably no

Hukka19:07:43

Do you have stats from npm or clojars on how widely used it is?

Hukka19:07:58

(I have no idea if those services collect any data like that)

thheller19:07:02

both seem widely unreliable

thheller19:07:06

but its growing steadily

Hukka19:07:19

I suppose it is quite difficult to convert those into distinct users, as someone might be using a cached version for a year, never updating, and someone else might have a clean CI build after every line (committed separately)

thheller19:07:04

yeah I don't care much about those numbers. new people keep ending up here and thats a good enough metric for me.

Hukka20:07:23

Ugh. The experience of starting a dev mode with material-ui is... bad. It's downloading many small files, so it takes over a minute to get all of those

Hukka20:07:42

I'm afraid it's also a transient dependency, so there's not much I can do from within my own code

thheller20:07:47

yeah I have been thinking about grouping all files from one lib into one file

Hukka20:07:02

And it looks like leaflet is somehow broken šŸ˜ž The map tiles are really wonky. No errors, though

Hukka20:07:32

Might have to try to make a minimal example without any plugins, since the project uses several and the new imports might play havoc

thheller20:07:13

IIRC leaflet was leaking some globals which were creating issues

Hukka20:07:15

Leaflet and its plugins are far from the best behaving modules in npm, instead relying on global vars even though they export themselves in name

Hukka20:07:34

Not only leaking, but designed to do it...

Hukka20:07:03

Before I was just loading the dist files in the meta section, and then referring them via js/L

Hukka20:07:01

Tomorrow, then

Hukka20:07:37

@thheller Bundling per package/import would be great

Hukka20:07:05

I didn't count, but there are hundreds, if not thousands of tiny files inside material-ui

Hukka20:07:17

Every material icon, for example

thheller20:07:24

yeah gotta figure out the best way to deal with source maps first

thheller20:07:44

yeah some js libs are absolute an absolute nightmare

thheller20:07:54

I think antd has something like 900 files as well

Hukka20:07:14

Is there a workaround? Perhaps refer directly to the dist file, if the npm lib has that?

thheller20:07:36

I looked at this recently since I want to build something similar for shadow-cljs

thheller20:07:59

I don't get this pattern

thheller20:07:22

hundreds of these index.js files that just re-export

thheller20:07:59

yeah you can load the dist file directly but that includes everything again. you really want to only include the components you actually use

Hukka20:07:08

Seems like the @material-ui/icons has so many files that unpkg times out showing the contents >_<

thheller20:07:53

yeah its a bit crazy

Hukka20:07:09

I know I do, and I'll try tomorrow if it makes a difference. I'm hoping my fingers crossed that the other packages I'd like to use don't just import the whole iconry wholesale

Hukka20:07:41

Oh well, it was just four files. Trying it now, though it's soon midnight

thheller20:07:42

if you can only (:require ["@material-ui/icons/Add" ...]) instead of @material-ui/icons and then icons/Add

thheller20:07:35

if you include the icons index it'll pull in every single icon and not a single one will be removed by DCE

thheller20:07:31

the whole library is really really bad IMHO. its just one mess after the other.

Hukka20:07:20

Oh, $Deity, yes, the mui addons I have refer to single icons too

Hukka20:07:48

I guess they rely on the DCE that does work pretty well in the webpack world

Hukka20:07:27

And if they did bundle them into bigger files, I suppose it would be even harder to use just a few

richiardiandrea23:07:31

hello folks, is there a switch or a closure define to detect if I am in watch/repl mode in shadow?

richiardiandrea23:07:48

I want to avoid executing the -main with target :node-script

richiardiandrea23:07:56

which happens when I watch and then launch node my-script

mitchelkuijpers07:07:25

The problem was that our build server had npm install --no-package-lock but that did not only not create a new package-lock.json but it would also ignore it... Now we updated node and changed the command to npm ci and life is great again ^^