If ClojureScript is, the shadow would be a good tool.

Is there a way to declare NPM dependencies in libraries so that those deps will be included in dependant projects that depend on those libraries ? i.e. how to declare transitive NPM deps without having to duplicate it in every dependent project ?

@superstructor you can include a deps.cljs with {:npm-deps {"foo" "version"}} in your .jar

Ok thanks, but that will just issue a warning not actually provide the deps ? Is there a good way to also provide transitive deps ? @thheller

hmm?

shadow-cljs will run an npm install on the packages listed in deps.cljs on startup? (if they aren't already in package.json)

unsure what you mean about a warning?

Is that behaviour any different between using the Node.js shadow-cljs CLI and running via lein with lein run -m shadow.cljs.devtools.cli and :lein true ?

no

which warning are you refering to? makes things easier if I know what you are talking about 😛

Ah I see it is the difference between :npm-deps and :foreign-libs ? For example

The required namespace "react" is not available, it was required by "day8/re_frame_10x/inlined_deps/reagent/v0v8v0/reagent/core.cljs".
The namespace was provided via :foreign-libs which is not supported.
Please refer to  for more information.
You may just need to run:
  npm install react

:foreign-libs is a different mechanism. has nothing to do with :npm-deps

you do need a deps.cljs at the root in your distributed .jar file

Ok I think that is clearer now. So that is for libs, but for apps themselves you still need to have your own mechanism for npm deps like package.json ?

yes that mechanism is for libs (which you asked about 😛)

Ok awesome, that is clear. Thank you! 😄

In the user guide there is only one mention of :npm-deps. May I ask where the source is and if you accept pull requests on docs ? I also wanted to add :external-config to the compiler options which is missing.

:external-config is not a thing in the compiler. it is purely community convention?

docs are here https://github.com/shadow-cljs/shadow-cljs.github.io

Not sure. :compiler-options { :external-config {:devtools/config {:features-to-install [:formatters :hints]}}} does work for cljs devtools.

yes but that is done by cljs-devtools by directly looking at the compiler-options

My bad, you are correct. After looking it up I see it does (get-in @cljs.env/*compiler* [:options :external-config :devtools/config]). :thumbsup:

yeah as far as CLJS is concerned there is only compiler-options

but it is a map which you can add anything to in theory

(shadow-cljs actually respects the key with regards to caching but thats it)

it is only under the :external-config key because of figwheel which had "strict" validation of the compiler-options map

and would complain about unknown keys

so they added the "convention" for external-config and tooling-config

but its purely on the library side

I see. Thanks I wasn't aware of that history.

One more question re earlier :npm-deps / :foreign-libs. For something like re-frame that needs to maintain compatibility with multiple build systems would it be recommended to simply declare both ?

IIRC there are open issues about that which I commented on?

basically just include a deps.cljs

hmm can't find it. maybe not.

just using cljsjs/react etc works but is helped by including a deps.cljs with :npm-deps

basically as far as shadow-cljs is concerned just include a deps.cljs

I thought that was actually added a while ago but I might be wrong

I can't see one in reagent or re-frame. react itself is probably a reagent PR to be more accurate. That explains how things should be going forward so it is clear, thank you! 🙂

Are you familiar with the lein-shadow plugin @thheller?

if that is the luminus thing yes

This one: https://gitlab.com/nikperic/lein-shadow

yeah that one

In my opinion, the implementation has too much coupling of different tasks/concepts, but regardless my follow up question is: Currently it gets :npm-deps out of project.clj. Might it make more sense from a design perspective for such a plugin to get that out of deps.cljs directly ?

Because then 1. apps and libs using lein could both declare npm-deps in the same way; and 2. when working directly in a libs repo, there would otherwise be duplication of deps between project.clj and deps.cljs.

wasn't aware it was doing anything with :npm-deps at all

apps should just be using package.json IMHO

libs should use deps.cljs

the problem is that npm is a total mess

Yep it generates package.json from :npm-deps key in project.clj. A bit of a hack, but it works.

so you NEED to hold onto your package.json AND the lockfiles (eg. package-lock.json) if you want anything remotely reliable

so anything that bypasses using npm directly is going to mess things up

Re your comment libs should use deps.cljs - If you are working directly in a libs project (in this case, it behaves like an 'app') you would need to duplicate deps between both package.json and deps.cljs ?

this is NOT maven where you can somewhat rely on dependencies not chaning unless you change them

Yep, correct. But if you choose to serialise package.json as deps.cljs; i.e. it is repeatable. Then it is OK ?

yes for libraries I maintain a package.json for development

and then just manually copy over deps.cljs when needed

well "repeatable" is questionable

Ok, so I guess that answers my question. That if you want to do that manual step automatically (ala lein-sahdow) :npm-deps should be taken from a deps.cljs not from project.clj.

It sort of works and I don't have a better solution is what I'd say

problem with taking it from deps.cljs is that you need to run a JVM with the full classpath to get them

which is much slower than just taking a :npm-deps map

I think that apps should maintain a package.json. period. don't outsource it to anything else.

it is annoying enough as it is. don't make it worse.

libraries with deps.cljs is problematic but works ok-ish until you run into version conflicts

then it becomes bad again

I honestly have no solution to these things so I'm all ears for better solutions

Thanks. That's a good summary of the issues.

I think the distinction between apps and libs becomes less interesting once your a lib developer, because then you run the lib as an 'app' (i.e. not a transitive dep) and you end up maintaining a package.json there too.

So you end up duplicating deps in 2 files at every layer of your dependency tree, which does suck.

Not only duplicating between files, but duplicating between dependent projects.

as a lib developer you'll need to be much more careful with you declared dependencies anyways

so I think it is ok to manually maintain a deps.cljs

again ... this isn't maven. options for managing dependencies are much more limited

for apps ... you will eventually need to maintain your own package.json

there is simply no way around that

Would it be worth considering shadow-cljs being responsible for managing/generation of package.json based on deps.cljs as a standard or is that a space you don't want to go ?

I'm pretty close to completely replacing npm and just downloading and managing dependencies manually

but no. the deps.cljs handling shadow-cljs currently does is all I'm willing to do

that is: read all found deps.cljs with :npm-deps on startup, do a best effort conflict resolution, run npm install the-package@version if the package is not in package.json already

everything else SHOULD be in package.json

otherwise we need to replicate every single tooling related thing npm does

which is just silly

Ok thanks, it is clear what the current extent of responsibility is :thumbsup: Yeah, if you choose to replace npm completely that would obviously avoid the mess and solve the issue, but npm does a lot of stuff. Its questionable how much of that CLJS users would actually need though for shadow-cljs to be viable.

As-in, if shadow-cljs did ever download/manage deps directly itself, I'm not sure it would actually need to cover 100% of npm's feature set related to that.

But yeah, I get its also a big scope increase, best avoided if possible.

the reason I haven't done that already is because it would only be practical for browser builds

as that is the only thing that doesn't actually need the node_modules folder structure

but most things do actually need that stuff (eg. node, react-native) so if we were to manage dependencies we'd have another system to worry about deps

so yeah I'm probably never touching that although I have thought about it a lot

Yeah that would be a formidable maintenance burden.

my recommendation if you want to avoid the npm horrors as best you can: use package.json, managed via npm install etc. and keep the lock files in version control

version ranges in npm are a nightmare and everything uses them

so there is no guarantee that 2 people running npm install the-thing will end up with the same dependencies

Yep, that's burned me before.

https://npm.github.io/how-npm-works-docs/npm3/non-determinism.html

things have gotten a bit better since lock files

but shit still happens

Yep, software is hard.

Thanks again for all the insights. Much appreciated. If I think up any possibly reasonable solutions to this I'll definitely ping you.

yeah combining to two different package managers is not that easy

Absolutely, its a tough position to be in. Overall, I want to say despite that shadow-cljs is amazing work and a huge leap forward in CLJS development. Its a good problem to have! It wasn't long ago that every time I needed a new JS dep it required a PR to CLJSJS!

yeah. although sometimes I wonder if we should maybe avoid using most JS deps 😉

I'm definitely at a stage where I try to avoid them as much as possible

too many times were I try to upgrade one tiny dep with a minor version bump and everything breaks 😛

and things seem to be getting worse not better in that regard

hope that webpack5 will bring back some order to the chaos 😛

Yeah, its a blessing and a curse. Unfortunately its non-trivial to replace all the JS things with CLJS things.

The set of functionality available in JS is always going to be greater than CLJS, so interop is a key part of CLJS's strength just like Clojure and the JVM w/ Java libs.

yeah absolutely. it is the reason I'm in this in the first place

but there are still a bunch of widely used npm packages that aren't very high quality

so in general the "quality" of JVM libs is much better (whatever that means)

True.

JS world is also still rapidly changing so I guess its something we have to live with for a while

to be fair to JS though they need to solve things that JVM mostly doesn't care about

eg. code size

Yep. Its nice to be insulated mostly from that churn by abstractions in the CLJS world, but the interaction at the interop/deps layer is going to be a pain point for awhile.

It'll be interesting to see how things play out. Looks like JS is becoming very much a compile target, esp with asm.js, TypeScript etc.

yeah, its probably be going to completely different (again) in a couple of years

npm isn't even 10 years old after all 😛

So a selling feature of CLJS would also be reliability of dependencies?

I started a PR on reagent to add deps.cljs as per above - https://github.com/reagent-project/reagent/pull/441 @thheller