Fork me on GitHub
#shadow-cljs
<
2018-08-30
>
flyboarder00:08:33

I give up, it builds just fine in docker container

flyboarder00:08:40

so something is wrong on my mac

thheller07:08:41

@flyboarder did you verify if app still has all the properties you expect? I know its easy to suspect shadow-cljs doing something wrong (and it might) but its just as likely that you accidentally upgraded something else and app just isn't what it used to be anymore?

thheller07:08:48

thats the first thing I'd verify

mhuebert12:08:44

Are there any known reasons how a browser REPL gets into a state where it returns [:did-not-find some.namespace] for namespaces that are part of the build?

thheller12:08:32

calling in-ns before require might do this?

thheller12:08:54

ie. you are trying to in-ns a ns that wasn't compiled yet

mhuebert12:08:34

maybe my flow is wrong, I was trying something new, starting with (shadow/watch :browser {:autobuild false}) so that it doesn’t always recompile whenever I tab to the browser, and then evaluating blocks or running “Load file in REPL” from intelliJ

thheller12:08:53

sounds good to me. maybe you just pressed switch-to-current-ns before load-file-in-repl?

mhuebert12:08:58

i’ll see if I can figure out exactly how to reproduce the state.

thheller12:08:22

switching itself doesn't load anything so you must either load-file or require first

mhuebert12:08:00

all of the namespaces are part of the original compile,

mhuebert12:08:15

but at some point i become unable to enter some of them

thheller12:08:21

note that the :did-not-find is a placeholder. I need to check what the standard CLJS REPL does in this place.

thheller12:08:45

but yeah it only happens in in-ns calls

thheller12:08:29

oh wait. I think I know how you can trigger it

thheller12:08:03

:autobuild false doesn't recompile on file change but the file change still invalidates the file so that it will be recompiled

thheller12:08:28

so yeah the ns disappears from the compiler state

mhuebert12:08:06

ok, yes now I can reproduce

mhuebert12:08:13

but even after I load the file, it still says :did-not-find

thheller12:08:33

load the file, change the file, call in-ns should be the steps

mhuebert12:08:01

I don’t quite understand that

thheller12:08:12

you load file in REPL your namespace

mhuebert12:08:15

so I currently have a namespace which was changed (marked invalid)

thheller12:08:34

without changes? what did you do?

mhuebert12:08:35

I can load file in REPL

mhuebert12:08:47

I just made an edit and saved

thheller12:08:21

ok that removes the ns from the compiler state

thheller12:08:26

if you then call in-ns it should fail

thheller12:08:36

but only AFTER editing it

thheller12:08:41

before it should work fine right?

mhuebert12:08:53

so it is now out of the compiler state

thheller12:08:59

yeah thats the watch in the background messing with the compiler state

mhuebert12:08:08

then I run load file in REPL, it appears to work — Loading cljs/src/spark_web/views/feedback.cljs... done

mhuebert12:08:14

but I still can’t in-ns

thheller12:08:39

that might be a race condition, did you wait long enough?

thheller12:08:14

load file once, make some changes to the file, save them, the load file again

mhuebert12:08:19

minutes, I mean it never works again, the only way I can use the ns again seems to be starting a new process

thheller12:08:37

hmm really?

thheller12:08:57

watch and REPL really don't play well together sometimes

thheller12:08:37

I gotta go eat something. I'll try to reproduce later.

thheller12:08:50

open an issue with the basic steps please

mhuebert12:08:06

is there a different way (other than watch) to compile such that the browser will connect to the shadow server for a REPL?

thheller12:08:04

shadow-cljs browser-repl but that probably has the same issue

mhuebert13:08:53

reproduction was not as clean-cut as I’d have liked

mhuebert13:08:51

I should also try with a fresh project to make sure it wasn’t anything to do with my particular build

thheller13:08:24

nah it makes total sense. the watch invalidating the files is just a bit overeager

thheller13:08:32

and just shouldn't do its stuff when autobuild is off

thheller13:08:40

so it doesn't interfere with the REPL

mhuebert13:08:27

so in a minimal example I am finding more reliable behaviour. maybe it also has something to do with how much time it takes to reload larger namespaces with a lot of dependencies? or could multiple changes trigger overlapping/interfering processes?

thheller13:08:54

there is definitely a timing element involved

thheller13:08:06

say you load-file and then save the file

thheller13:08:27

the save came after the load so it invalidates the file

mhuebert13:08:34

at some point in my small example I also saw things like this;

no source by id: [:shadow.build.classpath/resource "shadow_ex/core.cljs"]
{:id [:shadow.build.classpath/resource "shadow_ex/core.cljs"]}
ExceptionInfo: no source by id: [:shadow.build.classpath/resource "shadow_ex/core.cljs"]
        clojure.core/ex-info (core.clj:4739)
        clojure.core/ex-info (core.clj:4739)
        shadow.build.data/get-source-by-id (data.clj:147)
        shadow.build.data/get-source-by-id (data.clj:144)
        shadow.build.compiler/remove-dead-js-deps/remove-fn--14770/fn--14771 (compiler.clj:873)
        clojure.core/complement/fn--5391 (core.clj:1433)
        clojure.core/filter/fn--5614 (core.clj:2809)
        clojure.lang.LazySeq.sval (LazySeq.java:40)

thheller13:08:45

yeah I already know the problem

thheller13:08:09

it autobuild is off it'll have reset (meaning removed) a bunch of resources from the compiler state which compile usually adds in again

thheller13:08:23

but since compile doesn't happen its messed up

thheller13:08:36

just shouldn't do anything is autobuild is off

thheller13:08:40

problem solved 😉

thheller13:08:13

or maybe just remember to invalidate them but only do so before the next actual compile is triggered

thheller13:08:20

so it doesn't mess with the REPL doing stuff

thheller13:08:59

I'll think about it a bit

mhuebert13:08:36

sounds great

mhuebert14:08:22

also, not sure if this is already covered, but following a :did-not-find ... error, a few seconds later a REPL command timed out. message shows up

mhuebert14:08:44

might have expected that the :did-not-find would cause the command to stop/fail immediately?

flyboarder15:08:04

@thheller I agree with you, I believe this is being caused by classpath conflicts as I am getting many and noticed the closure compiler is from 2017

flyboarder15:08:21

So I think the version being used is faulty

flyboarder15:08:39

of the libraries not shadow

thheller15:08:43

@mhuebert nah the :did-not-find thing is really just a placeholder because I was undecided on how to deal with the situation. wanted to check what CLJS did but somehow forgot about it.

thheller15:08:53

@flyboarder I'm confused. I thought compilation worked fine but you were getting errors at runtime?

flyboarder15:08:30

I am it compiles without issues, but I suspect the version being pulled in is old

thheller15:08:22

so did you verify that app has the properties you expect it to have? that is the first and only question you should answer.

thheller15:08:53

none of the dependency issues matter at all if app isn't what you expect it to be

mhuebert16:08:05

just ran into a super weird issue trying to require auth0-js@latest. the minified version throws an error Cannot read property 'lib' of undefined when shadow requires it — partway into the file, a variable is undefined when it shouldn’t be. I can require the unminified version without any issues, so I guess it’s something to do with their minification

mhuebert16:08:17

it looks like they are using terser for minification

mhuebert16:08:37

should say: rollup w/ commonjs & terser

thheller16:08:59

what would I need to call to try this?

thheller16:08:26

(require '["auth0-js" :as x]) (x/???)?

mhuebert16:08:21

you don’t have to call anything - just require it

mhuebert16:08:28

it fails when evaluating the script

thheller17:08:49

uhm I only get that when I require it in the REPL? inside a normal build it seems to work?

mhuebert17:08:57

hmm I am running (shadow/compile :browser), and then opening the page

thheller17:08:04

version "9.7.3"?

mhuebert17:08:06

I don’t get the error if I shadow/release

thheller17:08:58

I do get the error if I just do shadow-cljs browser-repl and the require

thheller17:08:15

but it works in the browser build I use for testing

mhuebert17:08:46

i’m using a fresh/empty project

mhuebert17:08:16

are there any differentiating options around how code is evaluated when required?

thheller17:08:00

the REPL has some extra things but thats not active with just compile

thheller17:08:19

ok getting it in an empty project too

thheller17:08:05

hehe the minified code makes this tough to debug

thheller17:08:02

what the heck is this doing createCommonjsModule(function(module, exports) {

thheller17:08:06

thats just nuts 😛

thheller17:08:22

ok there is definitely something very odd happening

thheller17:08:37

(undefined.algo = {})) is in the transformed code but not in the source

thheller17:08:56

it might be closure just getting confused by the code. it might be my JS processing. no clue what is happening

thheller17:08:08

core = createCommonjsModule(function(module, exports) {
        var CryptoJS;
        module.exports = ((CryptoJS =
          CryptoJS ||
          (function(Math, undefined) {
            var create =
              Object.create ||
              (function() {
                function F() {}
                return function(obj) {
                  var subtype;
                  return (
                    (F.prototype = obj),
                    (subtype = new F()),
                    (F.prototype = null),
                    subtype
                  );
                };
              })();
            undefined = {};
            var C_lib = (undefined.lib = {}),
              Base = (C_lib.Base = {
                extend: function(overrides) {

thheller17:08:17

that has to be some of the weirdest JS I have ever seen 😛

mhuebert17:08:35

yeah, I was digging through that too. super weird!

thheller17:08:36

I guess its time to investigate this some more https://github.com/thheller/shadow-cljs/issues/173 😉

thheller17:08:00

minifying already minified files isn't all that great to begin with

mhuebert17:08:02

I wondered if maybe there were collisions with global vars, as I’ve run into that a bunch of times when mixing cljs code with other projects

mhuebert17:08:28

(to figure out what to add as global: externs)

thheller17:08:57

JS libs should never create global vars

thheller17:08:18

haha I think I know whats happening

thheller17:08:26

there is an actual function definition that uses undefined as a binding name function(Math, undefined) {

thheller17:08:50

closure seems to get confused by that and starts using it all over the place

thheller17:08:20

I'm surprised that undefined is even a legal binding name but this is JS so I expect nothing less I guess

thheller17:08:58

need to see if I can reproduce this with just closure. looks like a bug on their end

richiardiandrea17:08:02

question, do I understand correctly that a top level :clojure-defines in a build will be used for both :dev and :release?

thheller17:08:01

@mhuebert gotta love the JS world. you can temporarily fix it by setting :js-options {:variable-renaming :local}

thheller17:08:19

its definitely the closure compiler getting confused by the undefined

thheller17:08:57

I disabled variable-renaming by default to fix an issue in another library and making debugging a bit easier in general

thheller17:08:32

with variable renaming closure will change the function signature to function(a,b) { so the undefined is gone and nothing left to get confused about

mhuebert17:08:24

that is genuinely weirder than I even expected

mhuebert18:08:34

do you know why it only affected the minified version?

mhuebert18:08:47

as the non-minified version also has undefined there

mhuebert18:08:13

that :js-options setting works for me

thheller18:08:14

no idea why the non minified version doesn't have this issue. Maybe another function wrapper is added or so that makes it go boom

thheller18:08:15

module.exports = ((CryptoJS =
        CryptoJS ||
        (function(Math, undefined) {

thheller18:08:24

/**
		 * CryptoJS core components.
		 */
		var CryptoJS = CryptoJS || (function (Math, undefined) {

thheller18:08:34

prettified min vs non-min

thheller18:08:12

definitely changed in structure

thheller18:08:40

I don't even know how to begin describing this problem to the closure devs

thheller18:08:58

can't reproduce it with standard compiler since it doesn't allow turning variable renaming off

urbanslug22:08:05

Anyone know how I can have shadow pass this arg to clojure -Sdeps '{:mvn/local-repo ".m2/repository"}'

thheller22:08:28

:local-repo ".m2/repo..." in shadow-cljs.edn

thheller22:08:00

or do you use deps.edn and want it to use that? then there is no way other than calling clj -Sdeps '{:mvn/local-repo ".m2/repository"}' shadow.cljs.devtools.cli release app

urbanslug22:08:21

Yeah I think I'll have to call clj separately then

urbanslug22:08:42

I want it to only happen on CI and not disrupt dev

urbanslug22:08:56

the default is $HOME/.m2/repository but I want it to be ./m2/repository

urbanslug22:08:05

but only for CI

urbanslug22:08:21

whcih does build tests and release

urbanslug22:08:31

so for all 3 calls to shadow-cljs

thheller22:08:17

you can make an issue for this. happy to add something that makes this possible.

urbanslug22:08:22

I tried messing with the value of HOME to trick it but that didn't work haha

urbanslug22:08:14

There are a couple of issues here tbh. Are you talking about an issue regarding not being able to pass args to clj

urbanslug22:08:39

and there are issues like clj can't take that path as a var imo it should

thheller22:08:57

well if you are using deps.edn anyways I'd suggest just using clj directly.

thheller22:08:06

you gain nothing by going through shadow-cljs

urbanslug22:08:39

well I gain the convenience of "shadow-cljs release app;" which I'd have to figure out what it's actually calling

thheller22:08:34

replace shadow-cljs with clj -m shadow.cljs.devtools.cli

thheller22:08:38

thats what it calls

thheller22:08:54

clj -m shadow.cljs.devtools.cli release app

urbanslug22:08:38

ok let me try it

urbanslug22:08:56

Do you still feel there's an issue here?

thheller22:08:49

not if you use deps.edn

thheller22:08:47

also thought about "merging" configs a bit but generally not a fan

thheller22:08:12

but there should be an easier way to tweak some stuff for CI and such