Fork me on GitHub
#shadow-cljs
<
2019-02-28
>
anmonteiro10:02:36

@thheller hey thomas, me again

anmonteiro10:02:54

We’ve pushed and reverted code splitting to prod about 3 times now

anmonteiro10:02:27

note that all I’m gonna say is very empirical, as I can’t get a clear repro

anmonteiro10:02:55

we’re seeing this weird issue where apparently everything seems to work on a merged commit, but whenever someone merges another commit to master and we deploy the bundles break in subtle, horrific ways

anmonteiro10:02:41

my current hypothesis is that symbols are not “stable” (?) across different compilation runs?

thheller10:02:49

do you "mix" compilation results?

thheller10:02:00

meaning one module from a previous build reused in another?

anmonteiro10:02:11

I certainly hope not

thheller10:02:16

then yes symbols are not exactly stable

thheller10:02:30

do you use :module-hash-names?

anmonteiro10:02:36

nope, what’s that?

anmonteiro10:02:52

checking the guide

anmonteiro10:02:08

also, could :fn-invoke-direct true be weird here?

thheller10:02:35

otherwise cache might get in your way in that a user may have an older version of a module cached and mix it with a module from another build

thheller10:02:37

which will not work

anmonteiro10:02:00

I don’t think we need that

anmonteiro10:02:17

Buck generates its own hash out of compilation and we have “immutable compilation artifacts”

thheller10:02:37

ok but how do you use the hash?

anmonteiro10:02:44

to serve the bundle on the server side

thheller10:02:54

how though?

anmonteiro10:02:04

the Buck SHA gets generated after compiling the assets

anmonteiro10:02:16

we use to SHA as the path to upload it to the CDN

thheller10:02:34

yes but how do you get the hashes into the module loader data?

anmonteiro10:02:47

we embed it in the HTML as transit

anmonteiro10:02:55

we read that from the JS, and call shadow.loader.init

thheller10:02:15

ok so you set the shadow$modules var manually?

thheller10:02:26

and not inject it?

anmonteiro10:02:28

no, I call shadow.loader.init

thheller10:02:46

that only supports a prefix?

thheller10:02:56

ah ... the path

anmonteiro10:02:13

is there something I’m missing?

thheller10:02:24

ok yeah thats fine

thheller10:02:52

so it is guaranteed that all files from one build are in the same directory?

thheller10:02:04

and NEVER re-used or shared between builds?

anmonteiro10:02:08

note that my assumption could be wrong too

thheller10:02:12

> my current hypothesis is that symbols are not “stable” (?) across different compilation runs?

thheller10:02:16

why would that matter then?

anmonteiro10:02:16

yes that one

thheller10:02:46

symbols are only "stable" between builds if you preserve the .shadow-cljs/builds/<your-build-id>/release directory which you probably don't do

anmonteiro10:02:46

ok let me establish the symptom first

thheller10:02:11

stable as in cljs.core.assoc will be assigned the same short name in between builds

thheller10:02:21

and not xT in one and fY in annother

anmonteiro10:02:34

for code splitting we generate a thunk like you do in shadow.lazy

thheller10:02:36

but that really only matters to increase cache hits

anmonteiro10:02:45

in fact your recent shadow.lazy addition is very similar to what we do

anmonteiro10:02:54

which validated our approach somehow 😛

anmonteiro10:02:48

and what I think I’m seeing (again, I don’t get a clear repro so some things are very subjective) is that the symbol I embed in the thunk cant be found

thheller10:02:03

so the symptom with "mixed" code would be that module-b was compiled against module-a where cljs.core.assoc was named xT. Another build however assigned xT to cljs.core.conj. Now the module-b is suddenly calling cljs.core.conj in places where it expects to call cljs.core.assoc. Obviously breaking in horrific ways.

anmonteiro10:02:11

but all I get (for now) is undefined is not an object (evaluating 'M.h')

thheller10:02:05

did you try compiling with shadow-cljs release your-build --pseudo-names and check what that is?

anmonteiro10:02:14

I did try with pseudo-names

thheller10:02:16

or lookup the original name in case you have source maps?

anmonteiro10:02:29

and couldn’t repro, or badly

anmonteiro10:02:36

I think I got 1 repro with pseudo-names

anmonteiro10:02:46

and it said cst$sym$... doesn’t exist

anmonteiro10:02:50

something from the constants table

thheller10:02:59

shadow-cljs doesn't use the constants table

anmonteiro10:02:16

do you use cst$sym names at all?

thheller10:02:18

instead it optimizes the constants using a closure compiler pass

thheller10:02:40

so they can never be missing really 😛

anmonteiro10:02:42

I saw cst$sym$$props__auto__NNN for sure

thheller10:02:55

yes of course that are the generated constants

thheller10:02:43

hmm do you use js->clj on random JSON data?

thheller10:02:13

or does the page include other global JS that may override symbols from the CLJS compilation?

thheller10:02:45

it is not on by default for multi module builds

anmonteiro10:02:58

should I be using :output-wrapper explicitly?

anmonteiro10:02:05

I thought it was the default

anmonteiro10:02:32

@thheller so that would explain why I can’t repro in dev / staging

anmonteiro10:02:47

bc we have a bunch of external analytics scripts running in prod

thheller10:02:01

yeah that is probably it then

anmonteiro10:02:42

that could certainly explain why we don’t see it sometimes

thheller10:02:44

google analytics is a common offender when replaces something from the build

anmonteiro10:02:46

but then we see it in the next build

anmonteiro10:02:04

I’ll try that, thanks for help!

anmonteiro10:02:18

@thheller do you have a donation page for Shadow?

thheller10:02:36

working on it 😛

🎉 10
anmonteiro10:02:41

we’re putting aside some money for CIDER right now

anmonteiro10:02:54

could also set aside some budget to help out Shadow

anmonteiro10:02:06

@thheller this is important. Ping me directly when you’ve set it up please.

👍 15
thheller14:02:51

Since you all keep asking I finally started setting up my Patreon: https://www.patreon.com/thheller

👏 20
thheller14:02:28

I have no clue what I actually should do there so suggestions are very welcome

anmonteiro14:02:56

@thheller btw broken again with :output-wrapper…..

anmonteiro14:02:14

I’m sure this is my own fault but I’ve been at it for so long that I’m running out of clues

thheller15:02:02

@anmonteiro did you try source maps? does that provide any hint at all?

anmonteiro15:02:19

source maps are very confusing

thheller15:02:23

do you js->clj at all?

anmonteiro15:02:26

and don’t indicate the right lines

anmonteiro15:02:32

maybe because of macros?

anmonteiro15:02:44

js->clj where, for example?

anmonteiro15:02:53

like, what would it impact?

thheller15:02:48

js->clj has this problem with misidentified fast-path protocols

thheller15:02:08

there should be an open issue about that, let me see if I can find it

thheller15:02:47

it mostly shows if you use js->clj on JSON data that has short property names with numeric values. eg. {"x":4}

jeremys15:02:01

hello, quick question, I am trying to use shadow-cljs from a repl. When I use shadow.cljs.devtools.api/watch :a-build I get a “missing instance” error. From what I understand the exception comes from shadow.cljs.devtools.server.runtime/get-instance. Do I need to start something before using the watch fn?

anmonteiro15:02:08

hrm we do use js->clj in some places, but I don’t think they’d be in this code path

thheller15:02:12

@anmonteiro the problem shows in extremely weird ways. like showing errors on completely unrelated parts of the code that should not be called at all

jeremys15:02:14

@thheller oh ok thanks mate!

anmonteiro15:02:26

@thheller that’s what I’m seeing

anmonteiro15:02:30

> showing errors on completely unrelated parts of the code that should not be called at all

anmonteiro15:02:53

but I can’t repro locally

anmonteiro15:02:02

even when serving the optimizd bundle

thheller15:02:05

do you test with the same data?

anmonteiro15:02:32

probably not

thheller15:02:03

I spent days trying to debug this in the past until I finally figured out what was causing it and stopped using js->clj for JSON data 😛

thheller15:02:33

but could be something else entirely of course. Just sounds very familiar 🙂

thheller15:02:53

but you said you saw the issue with :pseudo-names which probably rules this out since the issue can't really happen with that

anmonteiro15:02:04

I don’t know anything anymore

🤕 4
thheller15:02:33

just in case did you try running shadow-cljs check your-build? It might provide a clue

anmonteiro15:02:36

can I just use cognitect.transit to read JSON data?

thheller15:02:03

not generic JSON data no

anmonteiro15:02:06

@thheller is there a way to run check as a library?

thheller15:02:57

(shadow.cljs.devtools.api/check :build-id)?

anmonteiro15:02:19

can it be run in dev?

thheller15:02:44

its not as useful as I'd like it to be since it reports too many false positives but something it can find hidden externs issues

thheller15:02:24

it just does a release build and runs the GCC type checker over it

anmonteiro15:02:29

673 warnings

thheller15:02:04

hmm yeah it goes a little overboard on some things

hlolli17:02:04

js.js:132 shadow-cljs - failed to load 59 how can I find out what module is number 59?

thheller17:02:37

dev or release?

thheller17:02:15

hmm actually that info isn't exported anywhere. you can set :js-options {:minimize-require false}

thheller17:02:32

or source maps?

hlolli17:02:36

yes, it lists it in source maps, I get three "failed to load" messages with different numbers, but only one stacktrace

thheller17:02:02

hmm you don't get the error in dev?

hlolli17:02:01

yup, it's lazy-cache module, npm ls lazy-cache shows a prompt module that Im using, wondering if I should remove it...

hlolli17:02:10

so only release

hlolli17:02:55

ahhh ok, sorry totally my fault, for some reason, I'm loading a nodejs module, I have some code that I share between node and browser, and it didn't get DCE

thheller17:02:40

yeah. I actually implemented removing unused JS requires but that had very bad consequences 😛

hlolli17:02:24

did you keep that setting optional for brave users?

thheller17:02:14

no. removed entirely. may come back in a different form at some point, maybe as a warning of sorts.

thheller17:02:40

eg. ns foo.bar required "xyz" but didn't use it

hlolli17:02:45

but I can avoid this by seperating the common code better. This release did work when I compiled in November with same namespaces that required the same modules.

thheller17:02:48

I generally check if the build report is clean and doesn't contain stuff it shouldn't

hlolli17:02:40

it makes sense in my case that it's included, a bit, I'm requireing 2 functions from a namespace that requires the unwanted module, but these functions don't call the module.

souenzzo19:02:55

feature request:

(defn get-manifest
  [id]
  (-> (shadow/get-build-config id)
      :output-dir
      (str "/manifest.edn")))
Or maybe return manifest from (shadow/release :id)

thheller19:02:55

what part in particular are you looking for?

thheller19:02:44

you can use (-> (shadow/get-build-config id) (shadow/release* {}) (extract-whatever-info-you-need))

thheller19:02:03

release* returns the full build state after compilation which contains everything the manifest contains?

👍 5
thheller19:02:29

release doesn't return anything on purpose since its meant to be used from the REPL

thheller19:02:45

in which case returning the build state for example blows up the REPL since it prints forever

thheller19:02:54

thats the extracted manifest data. you can probably limit it to the stuff you actually need

thheller19:02:28

(or (::closure/modules state)
    (:build-modules state))

thheller19:02:57

only release builds will have the :shadow.build.closure/modules data

neupsh21:02:38

Hi, is there a easy shadow-cljs project/template to get started with creating addons/extensions for browsers like firefox and chrome?

thheller21:02:37

@neupsh it is still kinda undocumented but it works and is used by people. https://github.com/thheller/shadow-cljs/issues/279

thheller21:02:49

don't know of any template, forgot what the other project was but there was another open source one

thheller21:02:58

at this point it probably won't change much anymore and I just have to finish the documentation

👍 5