Fork me on GitHub
#shadow-cljs
<
2021-11-23
>
thheller05:11:46

@swiseman108 when using lein to manage dependencies you need to make sure you are using the correct versions of everything. so make sure you are using the proper versions for the shadow-cljs version you are using. I do not have any issues with the closure compiler and I don't have guava at all either

Stephen Wiseman16:11:30

I think part of what I’m trying to figure out is how we’d be able to know which dependencies to use. Previously I’d be able to see the dep tree of shadow. But since shadow moved to a newer version of the closure-compiler (that no longer lists external deps), it’s near impossible to figure out which deps may conflict with the closure-compiler (as we saw with guava).

Stephen Wiseman16:11:18

i.e a lein deps :tree now shows zero dependencies under closure-compiler-unshaded. However, the closure compiler code can and will still conflict with other dependencies in your project. Just as we saw by pulling in guava 16.

Stephen Wiseman16:11:56

I’m also somewhat new to this, so I could be misunderstanding something key here 🙂

thheller18:11:46

yeah the closure compiler is a bit weird in that area. their bundling methods sometimes look like they don't expect people to use the closure compiler like we do 😛

thheller05:11:09

or as I always recommend let shadow-cljs.edn manage your dependencies. you only gain headaches by mixing your CLJ dependencies with CLJS ones

thheller05:11:00

(or use profiles/aliases for backend/frontend)

Stephen Wiseman15:11:27

Thanks @thheller. Moving to shadow-cljs.edn for dep management is definitely in the plans. Separating out frontend and backend deps would also be a solution to this particular issue as the lower version of guava is only in the dependency tree of a backend dependency.

zalky16:11:34

Hi there, running into an issue with cljs hot-reloading and trying to identify the point of failure using shadow's verbose output. I have three namespaces a > b > c, with a macro defined by b. So far hot reloading a seems to work though changing a does not seem to reload transitive dependency c (which seems odd, but maybe there's some optimized decisions or something). However, if I introduce an additional dependency between c and a , changing a reloads c before b, which is confusing to me: c relies on the macro in b, and results in an undefined var warning. Are my expectations about what gets reloaded and when, off? Let me know if it would help to post a simplified version the actual requires.

thheller18:11:51

@zalky namespace reload always works by reloading the namespace that was modified plus all namespaces that have a direct require for it. nothing else by default.

thheller18:11:28

reload order should happen in dependency order but I don't quite follow your description. macros are kind of tricky since they can technically do everything, so without an actual example I can't say much

zalky18:11:39

@thheller, thanks, I think I see. So a really simplified version of what it looks like is the following:

(ns p.a)
(defn f
  [])
(ns p.b) ; .clj
(defmacro macro
  []
  '(p.a/f))
(ns p.b ; .cljs
  (:require [p.a])
  (:require-macros [p.b]))
(ns p.c
  (:require [p.a]  ; <-- this line causes p.c to be compiled before p.b
            [p.b]))
(p.b/macro)

thheller18:11:55

incorrect. that line causes p.c to be compiled at all when p.a is changed. without that it won't

thheller18:11:12

what causes it to compile c before b is whatever requires those

zalky18:11:13

Yes, my mistake.

thheller18:11:06

but why is that a problem in the first place? is the macro doing some side effecting stuff that is causing a problem?

thheller18:11:11

otherwise it really shouldn't matter at all?

zalky18:11:06

Are you asking in the sense of why do the undefined var errors matter?

zalky18:11:30

Warnings I should say.

thheller18:11:47

again without an actual reproducible example I cannot comment

thheller18:11:59

macros can do too much stuff to know what exactly you are describing

zalky18:11:13

That's fair, they can be. This particular macro just produces some runtime code inside component render functions. I don't think it does any compile time side-effects. Do you know what part of the system, shadow or the cljs compiler, that does the dependency graph resolution? ie: resolves the dependency order?

thheller18:11:04

fwiw I took your example code and things compile in the expected order (ie. modify a compiles a->b->c)

thheller18:11:49

the only way I can see this not happening is p.c using p.b without requiring it

thheller18:11:20

in which case yes all bets are off on the order part

zalky18:11:35

Ok, much appreciated, the actual code is part of a fairly large framework, and I wasn't immediately clear that only immediate dependents are recompiled. I think actual example has another namespace between p.a and p.b, and so p.b is not actually being reloaded in either case.

zalky18:11:25

p.c is indeed requiring p.b

thheller18:11:47

the reasoning behind only recompiling direct dependents is that in theory thats the only namespaces that can "break". macros kinda break that assumption in certain cases so absolutely possible that things behave weirdly but most of the time that is due to macros doing some side-effecty stuff looking at analyzer data or so

zalky18:11:40

Ah, hmm, the macros is indeed looking at &env to determine if a particular symbol is a bound local

thheller18:11:04

locals should be fine?

thheller18:11:33

ah if it calls resolve then that may trigger the warning, not the compile itself

zalky18:11:18

It does not call resolve, it just looks up whether a symbol is locally defined, and if it is not, provides a default binding.

zalky19:11:37

Can I also ask, during the compilation, I'm seeing output like the following:

1:49:52 PM cljs.1 |  -> Compile CLJS: a.cljs
1:49:52 PM cljs.1 |  -> Compile CLJS: c.cljs
1:49:52 PM cljs.1 |  <- Compile CLJS: c.cljs
...
1:49:52 PM cljs.1 |  <- Compile CLJS: a.cljsi
Where c depends transitively on a. Does it mean these compilation steps are an asynchronous process? Are there any concurrency concerns here that might produce an undefined var error?

thheller19:11:38

there are multiple threads compiling yes. I still won't comment any further without an actual reproducible example. guessing here is really pointless.

zalky19:11:09

Ok, understandable. I don't suppose there are any options in shadow to make that compilation synchronous? I didn't see anything in the docs, just thought I'd check.

thheller19:11:32

you can set :compiler-options {:parallel-build false} in your build config

zalky19:11:34

Ok, thanks, I'll see where I can get with that, maybe I can get a simple repro. Very much appreciate you taking the time to provide some pointers.

zalky20:11:22

@thheller, I put together a small https://github.com/zalky/shadow-repro. As best I can tell, it seems to me like a concurrency issue when it comes to macro compilation and transitive dependencies between recompiled namespaces. Let me know if I may have missed anything or could provide anything else that might be useful.

awb9921:11:10

I have a question on polylith builds (so multiple deps.edn projects with :local/root dependencies. Could it be that shadow-cljs does not pick up the deps.cljs of multiple projects in this case? So I have to create ONE big deps.cljs file. I assume that shadow-cljs is looking only into JARS, and since the local deps are FILES, it will not find them inside a jar. I have not noticed a similar behavior when using JAR dependencies on clojars for exampel.

awb9921:11:10

This issue does not happen, when the deps.cljs are in an empty namespace, so say in "src" of each project.

awb9921:11:31

However, in this case when multiple projects get ombined into a JAR, then of course the classpaths of that jars will conflict.