Fork me on GitHub
#shadow-cljs
<
2018-07-11
>
eoliphant13:07:18

hi, what’s a good way to handle config stuff that varies across environments? say dev vs production api endpoints,

hlolli13:07:45

@eoliphant I'm useing webpack with shadow since recently, and there I can do something like this in plugins, reading from environmental variables

new webpack.DefinePlugin({
            'DEVELOPMENT': JSON.stringify(process.env.NODE_ENV === 'development'),
        }),
then I can see if I'm in development mode
(def dev? js/DEVELOPMENT)
some plugin like this may exists with other means.

👍 4
eoliphant13:07:09

yeah, I was hoping for something ‘shadow only’ lol.

eoliphant13:07:21

I see this for the build itself https://shadow-cljs.github.io/docs/UsersGuide.html#_release_specific_vs_development_configuration wondering if I can use that plus the build hooks

hlolli13:07:16

nice! way better 😛

eoliphant13:07:55

sometime you just need to talk stuff thrugh lol. I literally read past this twice lol

👍 4
hlolli13:07:05

I've read past this more than twice, and finally it clicked now in my brain what it's used for.

thheller15:07:43

:closure-defines is a good option, you can also use https://shadow-cljs.github.io/docs/UsersGuide.html#_conditional_reading or combine both

👍 4
justinlee15:07:06

when I see this line <- Cache write: seekeasy/upload.cljs (4825 ms) does that mean it is taking 4.8 seconds just to write a single file to disk or does that include some kind of wait time for other writes too?

thheller15:07:48

4,8sec to write the Cache for that file only. does not include any other time. how big is that file?

justinlee16:07:27

it’s 35 lines of cljs code

thheller16:07:52

does it include a macro that inlines some gigantic datastructure?

justinlee16:07:56

this seems to happen over time

justinlee16:07:37

no macros are actually used in the file. it is just a single defn

justinlee16:07:04

i bet if i stop and restart the process it will be fast again

thheller16:07:41

thats weird. I have never seen this get slower over time. can you check how big .shadow-cljs/builds/<your-build>/dev/ana/seekeasy/upload.cljs.cache.transit.json is?

thheller16:07:58

before restarting please?

thheller16:07:08

thats way too large for a simple file

justinlee16:07:40

wtf there’s all kinds of crazy shit in here

justinlee16:07:58

there’s just pages and pages of this kind of stuff "^2I","^2D","^43","^44","^5W","^6R"

thheller16:07:18

its the cache data structure in transit

thheller16:07:34

can you send it to me please? might include a clue whats repeating

thheller16:07:13

thats normal transit encoding for keywords and other constants. not unusual.

thheller16:07:47

I suspect that the analyzer data is growing over time for some reason

thheller16:07:00

its supposed to get a reset when the file is recompiled

justinlee16:07:43

okay sent. it took me a really long time to remember how to show a freaking dotfile in mac haha

thheller16:07:22

thx checking it out. now you can try restarting the server and recompiling that file

thheller16:07:29

cache should be smaller then

thheller16:07:17

ok there is definitely something odd going on in there

justinlee16:07:18

i sent you the same file after restart

thheller16:07:25

just to rule out outside factors: do you use pure shadow-cljs.edn or lein/tools.deps?

justinlee16:07:37

pure shadow. no dirty lein for me 🙂

thheller16:07:49

do you use any other CLJ side stuff? cider, tools.namespace?

justinlee16:07:07

the only weird thing i have is ghostwheel

justinlee16:07:16

though this file doesn’t directly invoke any of that

thheller16:07:33

thats only spec related and doesn't mess with namespaces directly IIRC

justinlee16:07:02

:dependencies
 [[org.clojure/clojure "1.9.0"]
  [org.clojure/clojurescript "1.10.238"]
  [com.bhauman/spell-spec "0.1.1"]
  [expound "0.7.1"]
  [metosin/spec-tools "0.7.1"]
  [gnl/ghostwheel "0.2.2"]
  [org.clojure/test.check "0.9.0"]
  [reagent "0.7.0"]
  [secretary "1.2.3"]
  [venantius/accountant "0.2.3" :exclusions [org.clojure/tools.reader]]
  [funcool/promesa "1.9.0"]
  [cljs-ajax "0.7.3"]
  [com.rpl/specter "1.1.0"]]

justinlee16:07:13

and i just use cursive

thheller16:07:38

module$node_modules$react_dom$lib$reactProdInvariant
module$node_modules$react_dom$lib$ReactPropTypesSecret
module$node_modules$prop_types$factory
module$node_modules$react$lib$React
module$node_modules$fbjs$lib$invariant
module$node_modules$fbjs$lib$warning
shadow.js
module$node_modules$react_dom$lib$reactProdInvariant
module$node_modules$react_dom$lib$ReactPropTypesSecret
 module$node_modules$prop_types$factory
module$node_modules$react$lib$React
module$node_modules$fbjs$lib$invariant
module$node_modules$fbjs$lib$warning

thheller16:07:02

these are supposed to be distinct. checking why they are repeated over and over again

thheller16:07:40

yeah the clean file doesn't repeat them endlessly

thheller16:07:38

ah found it I think

sakalli16:07:16

hi *. n00b q's: decided to try to build an app on shadow-cljs today. used the

lein new shadow-cljs 8bs-vision-survey +reagent
template. according to the readme it has the following commands:
yarn install, yarn watch, yarn clean, yarn release
the shadow-cljs.edn file looks like this
;; shadow-cljs configuration
{:source-paths
 ["src"]

 :dependencies [[binaryage/devtools "0.9.7"]
                [reagent "0.8.0-alpha2"]
                [org.clojure/core.async  "0.4.474"]
                [com.pupeno/free-form "0.6.0"]
                [cljs-http "0.1.45"]
                [com.cemerick/url "0.1.1"]]
 
 ;; set an nrepl port for connection to a REPL.
 :nrepl        {:port 8777}

 :builds
 {:app {:target :browser
        :output-dir "public/js/compiled"
        :asset-path "/js/compiled"

        :modules
        {:main
         {:entries [vision-survey-8bs.core]}}

        :devtools
        ;; before live-reloading any code call this function
        {:before-load vision-survey-8bs.core/stop
         ;; after live-reloading finishes call this function
         :after-load vision-survey-8bs.core/start
         ;; serve the public directory over http at port 8700
         :http-root    "public"
         :http-port    8700
         :preloads     [devtools.preload]}
        }}}
got it working while developing and decided to deploy the app. i ran
yarn release
and deployed the public folder to a server. the app runs but only displays the placeholder page with "Shadow-cljs rocks!" - text. any pointers how to fix it?

thheller16:07:40

@sakalli any messages in the browser console?

sakalli16:07:05

ah yeah. something wrong with the path to the js. lets see if i can fix it. thanks.

thheller16:07:25

@lee.justin.m trying to figure out how this happens over time. do you have one watch running for very long times (days) or do you restart frequently?

justinlee16:07:22

one watch that goes for days

justinlee16:07:31

i rarely restart

justinlee16:07:54

and i’m pretty sure this takes a long time to manifest

thheller16:07:51

yeah it probably gets slower with every recompile

justinlee16:07:41

oh yea on a single recompile it went from 68314 to 77632

thheller16:07:56

or rather it gets slower for every file that is not recompiled

justinlee16:07:58

and i didn’t change anything except whitespace in one of its dependencies

justinlee16:07:52

i see, so i have certain utility files that everything depends on. when i touch those files, all of the cache files increase in size

justinlee16:07:12

when i just touch upload.cljs, just that file and its dependencies get bigger

thheller16:07:04

nevermind. of course its growing when I edit the file and change the contents 😉

thheller16:07:49

@lee.justin.m should be fixed in 2.4.20. pretty obvious bug when looking at it. to the degree that I can't figure out how caching works at all. that should really have invalidated the cache all the time.

sakalli16:07:52

ok. so got the path working but there is another error "SyntaxError: identifier starts immediately after numeric literal" referring to the line

<script>vision-survey-8bs.core.init();</script>
at the end of the index document. so no numbers allowed in function names?

thheller16:07:55

@sakalli that is javascript where - is actually minus. must "munge" the name to use _ instead. <script>vision_survey_8bs.core.init();</script>

justinlee16:07:32

The Six Stages of Debugging:

1. That can't happen.
2. That doesn't happen on my machine.
3. That shouldn't happen.
4. Why does that happen?
5. Oh, I see.
6. How did that ever work?

😂 12
thheller16:07:48

exactly 🙂

thheller16:07:25

@lee.justin.m thanks for the report and the files. would have taken ages to debug without.

sakalli16:07:38

is there a hack to munge it other than manually or fixing the whole app (which is quite some work since this name is in the directory structure and all over the app). very little js experience.

thheller16:07:22

@sakalli what do you mean? the CLJS compiler does that automatically. you just need to do it once in the HTML when calling the init function.

thheller16:07:17

when it doesn't work after changing the _ you just might be missing the important ^:export bit on the (defn ^:export init [] ...)

justinlee17:07:45

@thheller thank YOU for being such a freaking wizard. the fix appears to be working for me

justinlee17:07:36

now i think i’m going to have to ditch ghostwheel because it is basically has to be recompiled anytime i change any file. i’d rather have subsecond turnaround i think

thheller17:07:47

I only looked at it briefly. seemed nice but also like a lot of overhead 🙂

justinlee17:07:59

i’m kind of missing my .h files: they sure did make compilation faster

justinlee17:07:40

on a project of considerable size it must be very frustrating to iterate on utility files as the whole world has to be recompiled

thheller17:07:47

well fancy things like ghostwheel must cost something 😉

thheller17:07:30

yeah I really want to do some performance work but that takes sooo much time

thheller17:07:59

it bothers me already when I have to wait 3sec or so

thheller17:07:26

the fastest thing you can do is working at the REPL without live-reloading at all

justinlee17:07:30

i guess i’m just not sure why, for example, ghostwheel/core.cljc has to be recompiled when I change file A, which includes file B, which uses ghostwheel, even though B and ghostwheel don’t change

thheller17:07:37

since you control what gets reloaded and when

thheller17:07:49

but its also more manual and annoying at times 😉

justinlee17:07:50

i have fantasies about creating some kind of build system that loads everything up and creates a granular database of functions and macros and transcends the concept of a file. then we can all ascend into developer heaven

thheller17:07:55

why is ghostweel.core recompiled?

justinlee17:07:10

i don’t know. but it is recompiled everytime i touch any file

thheller17:07:27

hmm that shouldn't be?

justinlee17:07:58

oh well if that’s a bug then that’d be a good thing. i thought it was just something inherent to the messiness of macros

justinlee17:07:12

should i try to create a small repro?

thheller17:07:21

oh right. whole lots of magic macros. still shouldn't recompile ghostweel.core if you touch a namespace that requires it though

thheller17:07:52

or does it produce warnings?

gnl17:07:59

If I may chime in – ghostwheel really shouldn't be recompiled every time you just change a namespace which uses it, I would definitely consider this a bug, although not yet sure where. I don't think I've ever witnessed that, but I'll look into it on my end, see if I can reproduce it.

justinlee17:07:39

i’m trying to create a repro here too. likely is user error on my part

gnl17:07:58

Are you somehow directly referencing the actual ghostwheel source in your project – for example by adding the cloned repo to your source paths or something – or just using the clojars artifact?

thheller17:07:52

@lee.justin.m does this still happen in 2.4.20? I can totally see this happening with the older versions

justinlee17:07:47

okay yea this is easy to repro

gnl17:07:05

I think this one's on me.

gnl17:07:09

Gimme a few minutes.

justinlee17:07:22

thanks for looking into it!

gnl17:07:10

Hey, I'm as allergic to long recompile/reload times as anyone. 🙂

gnl17:07:18

@lee.justin.m Okay, try 0.2.3-SNAPSHOT

gnl17:07:14

This is embarrassing – I had ^:dev/always set on the tracing namespace (leftover from development) and ghostwheel.reporting+ghostwheel.core depend on it, so there's a whole lot of unnecessary work being done.

justinlee17:07:06

oh yea now its zippy. no ghostwheel recompiles anymore

justinlee17:07:41

took my recompile from >2s to <1s

justinlee17:07:02

and now that we’ve fixed the shadow-cljs bug, my 14s builds are less than 1s too 🙂

parrot 4
justinlee17:07:13

@thheller one thing: when i updated the version of ghostwheel in shadow-cljs.edn, it kicked off a recompile, but it didn’t actually download new artifacts. is that intended?

justinlee17:07:33

I had to restart to get new artifacts.

gnl17:07:57

@lee.justin.m I just went through the six stages of debugging you described above in like 15 seconds.

gnl17:07:02

That was a ride. 🙂

gnl17:07:28

I'll see about releasing 0.2.3 soon, cause this one's painful.

thheller17:07:14

@lee.justin.m recompile should only happen if the build config changed. only :dependencies should not trigger it. downloading new artifacts is only done of startup since doing that at runtime is way too complicated

justinlee17:07:22

okay cool. the recompile must have been because i hadn’t saved some other file because it isn’t doing it now.

thheller17:07:59

bbiab. gotta buy some food.

gnl22:07:53

@lee.justin.m – if you run into anything like this again, please let me know. Ghostwheel's runtime overhead should be zero and the recompile overhead virtually zero (the code generation here is really not particularly complex or expensive), unless you enable tracing and/or gen-testing on hot-reload, which will of course slow things down.

justinlee22:07:29

@clojurians.net cool thanks. i’m still not 100% on how the compiler works so i’m never quite sure if there’s an issue. it’s a real delight to deal with folks like you and @thheller so thanks for that

👍 4
gnl22:07:35

When you're not sure just ask on #ghostwheel first and if we can't clear this up quickly, you can open an issue on GitHub. I need eyes on this thing to get it solid, so feedback is appreciated.