Fork me on GitHub
#hoplon
<
2016-08-31
>
thedavidmeister09:08:22

hey, so i have kind of a dumb question

thedavidmeister09:08:30

i’m using boot watch to reload files

thedavidmeister09:08:34

but it only really works for cljs

thedavidmeister09:08:51

is that expected, that the clj side of things won’t reload on watch?

dm309:08:38

how do you expect it to reload?

dm309:08:45

restart the whole application?

dm309:08:56

or just recompile the code?

dm309:08:17

some changes can be recompiled to get the effect, some cannot

dm309:08:39

so usually it's up to the user to specify how and when you want to get the backend code reloaded

thedavidmeister09:08:06

well i think recompiling would be ok

thedavidmeister09:08:15

atm i change a function and nothing at all happens

thedavidmeister09:08:40

i can’t even stick in a prn to debug without waiting for a 70s+ boot command 😕

thedavidmeister09:08:51

@dm3 i think i’m missing something very basic 😉

dm309:08:30

I usually recompile stateless things directly from the editor as I work

dm309:08:58

emacs/vim/Intellij support connecting the editor to the nRepl process

thedavidmeister09:08:17

hmm, but it’s a web server

dm309:08:31

after changing something that is not recompileable, e.g. types that are part of a running application state

dm309:08:42

or things that got captured in the application state

dm309:08:06

I usually clojure.tools.namespace.repl/refresh-all

dm309:08:33

which computes the namespace dependency graph and reloads all of the namespaces where the dependencies got modified

thedavidmeister09:08:10

does that work with the code the web server would be using too?

thedavidmeister09:08:28

i’ve just been pasting (repl/refresh) into the repl as needed so far

thedavidmeister09:08:29

i suppose i’m confused because my boot task is like (comp (watch) (serve …)) so i had thought the watch would “re-serve"

thedavidmeister09:08:32

but it doesn’t work that way

thedavidmeister09:08:53

wait, i have an idea, lol

thedavidmeister09:08:45

no, i don’t think that worked

dm310:08:45

not sure about serve

thedavidmeister10:08:09

@dm3 i think i figured it out

thedavidmeister10:08:12

and i think it was very basic

thedavidmeister10:08:09

the serve task has a :reload true option

thedavidmeister10:08:39

yup, i wasted a loooot of time not having that turned on 😕

thedavidmeister11:08:06

time env BOOT_JVM_OPTIONS="-client -Xmx2g -Xverify:none -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled" boot -h

thedavidmeister11:08:17

i see a speedup for boot

thedavidmeister11:08:34

but when i put BOOT_JVM_OPTIONS="-client -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -Xmx2g -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -Xverify:none”

thedavidmeister11:08:42

into boot.properties nothing happens

dm311:08:05

where are the boot.properties?

dm311:08:26

also you don't need the quotes in the properties file

thedavidmeister11:08:56

well the file worked for other things, like BOOT_EMIT_TARGET

thedavidmeister12:08:02

also, is it normal that compiling cljs without any optimisations takes ~30 seconds?

dm312:08:43

depends on how much cljs you've got

dm312:08:02

does 30 seconds include Clojure startup time?

thedavidmeister12:08:51

it’s from the point boot says Compiling ClojureScript…

thedavidmeister12:08:11

the whole thing from boot dev to something i can load in the browser is more like 70s

thedavidmeister13:08:37

i’ve got maybe 1700 LOC in cljs

onetom13:08:48

@thedavidmeister yes, 30-70s is possible if it's a clojurescript project how fast can this project start for you (after the deps has already been downloaded)? https://github.com/addplus/hoplon-ui-reload-example

thedavidmeister13:08:17

i just get errors for that repo @onetom

onetom13:08:20

why are you pasting pictures instead of just text?

thedavidmeister13:08:03

the text is very long

onetom13:08:07

ah, sorry, i forgot hoplon-ui is not in clojars yet. here is a similar repo which is good for benchmarking: https://github.com/onetom/boot-reloadable-hoplon

onetom13:08:09

slack collapses text anyway and it takes less bandwidth to receive it.

onetom13:08:48

pair@dusty ~/g/o/boot-reloadable-hoplon> time boot dev
Starting reload server on 
Writing adzerk/boot_reload/init3346.cljs to connect to ...

Starting file watcher (CTRL-C to quit)...

Writing HTML files...
• index.html
Adding :require adzerk.boot-reload.init3346 to index.html.cljs.edn...
Compiling ClojureScript...
• index.html.js
Starting Vert.x on port 8000...
Elapsed time: 19.989 sec

^C
Stopping Vert.x on port 8000...
       32.18 real        73.49 user         3.40 sys

thedavidmeister13:08:08

titdm:boot-reloadable-hoplon thedavidmeister$ time boot dev
Starting reload server on 
Writing adzerk/boot_reload/init3306.cljs to connect to ...

Starting file watcher (CTRL-C to quit)...

Writing HTML files...
• index.html
Adding :require adzerk.boot-reload.init3306 to index.html.cljs.edn...
Compiling ClojureScript...
• index.html.js
Starting Vert.x on port 8000...
^C
Stopping Vert.x on port 8000...

real	0m26.508s
user	1m34.580s
sys	0m3.406s

onetom13:08:19

Processor Name: Intel Core i5 Processor Speed: 2.7 GHz Number of Processors: 1 Total Number of Cores: 4 L2 Cache (per Core): 256 KB L3 Cache: 6 MB Memory: 16 GB

onetom13:08:17

what helps with any bigger apps is the parallel compilation option

thedavidmeister13:08:42

yeah i turned that on

thedavidmeister13:08:46

but i still get this 😞

thedavidmeister13:08:52

Starting reload server on 
Writing boot_reload.cljs...

Starting file watcher (CTRL-C to quit)...

Started HTTP Kit on 
Compiling Hoplon pages...
• index.cljs.hl
Adding :require adzerk.boot-reload to index.html.cljs.edn...
Compiling ClojureScript...
• index.html.js
Writing target dir(s)...
Elapsed time: 59.284 sec

^C
real	1m23.689s
user	1m49.179s
sys	0m5.921s

onetom13:08:20

why are you writing to the target dir?

onetom13:08:42

you only need that when you are preparing a deployment artifact

dm313:08:36

also, do you have an ssd? 🙂

onetom13:08:07

also notice that you are starting both the frontend and the backend probably, so you load (and compile) a lot more libraries

thedavidmeister13:08:06

shaved about 10s

thedavidmeister13:08:10

Starting file watcher (CTRL-C to quit)...

Started HTTP Kit on 
Compiling Hoplon pages...
• index.cljs.hl
Compiling ClojureScript...
• index.html.js
Elapsed time: 54.853 sec

^C
real	1m14.745s
user	1m43.203s
sys	0m5.179s

thedavidmeister13:08:41

also yeah, ssd 🙂

thedavidmeister13:08:24

this is how i’m doing cljs

thedavidmeister13:08:27

(cljs :optimizations :none
          :compiler-options { :parallel-build true
                              :cache-analysis true})))

dm313:08:56

well, I've never even timed how long my projects take to start

dm313:08:12

because it happens just a couple times a week

thedavidmeister13:08:29

well i think it will happen to me less now actually

thedavidmeister13:08:38

now that i found that :reload true option to get the backend to reload

onetom13:08:40

Also if I use all those advance startup time saving options, I actually get the following:

pair@dusty ~/g/o/boot-reloadable-hoplon> time env BOOT_JVM_OPTIONS="-client -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -Xmx2g -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -Xverify:none" boot dev
Starting reload server on 
Writing adzerk/boot_reload/init3346.cljs to connect to ...

Starting file watcher (CTRL-C to quit)...

Writing HTML files...
• index.html
Adding :require adzerk.boot-reload.init3346 to index.html.cljs.edn...
Compiling ClojureScript...
• index.html.js
Starting Vert.x on port 8000...
Elapsed time: 13.981 sec

^C
Stopping Vert.x on port 8000...
       20.23 real        28.71 user         1.70 sys

onetom13:08:08

i only use this though:

pair@dusty ~/g/o/boot-reloadable-hoplon> echo $BOOT_JVM_OPTIONS
-Xmx2g -Xverify:none

thedavidmeister13:08:25

yeah i have all the things turned on

dm313:08:25

yep, I'd alias the params under something like alias boot-dev=env BOOT_JVM_OPTIONS...

onetom13:08:42

because the incremental compile can get optimized over time and gives us around 1second re-compile times

onetom13:08:34

if i have those startup time optimizations, my incremental compile time doesn't go below 1.5-1.8s

onetom13:08:43

which is just enough to feel uncomfortable with it

thedavidmeister13:08:53

my incremental compile is like 3-4s

onetom13:08:02

that sounds sluggish...

thedavidmeister13:08:38

yes, my cljs is really slow

onetom13:08:49

btw, the :cache-analysis true doesn't seem to affect the performance unfortunately

thedavidmeister13:08:37

any other things i can tweak?

onetom13:08:14

you should try this vertx based server too, using the boot-static package, but that would assume you have your backend served separately

onetom13:08:28

for that im still using boot-jetty

onetom13:08:53

what the spec of your processor?

onetom13:08:18

also you pressed Ctrl-C too early in your example. you should wait until the Elapsed time: ... appears

onetom13:08:41

can you check your cache sizes too in the system report, pls?

thedavidmeister13:08:09

ah i see, so you can have one boot task to fire up the backend and one to handle cljs?

onetom13:08:28

also with 8GB ram i can imagine that you might max that out and your cljs compilation process is forced to do swapping...

thedavidmeister13:08:26

Processor Name:	Intel Core i7
  Processor Speed:	2.6 GHz
  Number of Processors:	1
  Total Number of Cores:	4
  L2 Cache (per Core):	256 KB
  L3 Cache:	6 MB
  Memory:	8 GB

onetom13:08:31

pair@dusty ~/a/be> time boot dev
nREPL server started on port 63750 on host 127.0.0.1 - 

Starting file watcher (CTRL-C to quit)...

Starting #'sys/dev
Elapsed time: 1.604 sec

^C       17.02 real        43.73 user         1.53 sys

onetom13:08:54

and my recompile + reload times are like this:

#'sys/dev:refreshing
Unloading: (app.profile-test app.users-test app.core-test app.schema-test sys app.schema app.core app.users)
Reloading: (app.users app.core app.schema sys app.schema-test app.core-test app.users-test app.profile-test)
Elapsed time: 0.689 sec

onetom13:08:42

which is actually slow because those *-test namespaces shouldn't even be reloaded really i think...

onetom13:08:09

Independent Deployability FTW as Uncle Bob would say 🙂

thedavidmeister13:08:10

would that be ok if i use sente?

onetom13:08:49

now, that i don't know 🙂

onetom13:08:13

imean it should work of course but i haven't tried it myself yet

thedavidmeister13:08:04

aight, i guess i’ll put some more tickets up

thedavidmeister13:08:08

thanks for the help

thedavidmeister13:08:18

anything to get these load times down is useful

thedavidmeister13:08:27

it is getting to the point where it’s getting in the way

onetom14:08:03

so i don't know what the pitfalls would be. in case of using castra, i had to specifically use [jumblerg/ring.middleware.cors "1.0.1"] for the CORS support

onetom14:08:03

also that backend reloading is implemented by the stuart sierra REPL flow which im using via danielsz/boot-system

onetom14:08:53

the problem i found with panderios/boot-http is the it's just using the jetty server built-in reload functionality, which is only triggered during serving a request

onetom14:08:04

which means that request is still served by the old code

onetom14:08:06

so you are experiencing an earlier version than your latest change

onetom14:08:14

which is fucking misleading of course...

thedavidmeister14:08:20

that is annoying

thedavidmeister14:08:38

so its always one behind?

onetom14:08:13

so i switched to

<dependency>
            <groupId>tailrecursion</groupId>
            <artifactId>boot-jetty</artifactId>
            <version>0.1.3</version>
            <scope>compile</scope>
        </dependency>
for a long time ago

onetom14:08:52

yes, it was always behind. it might have been an old bug but at that time it seemed like an architectural issue of boot-http

thedavidmeister14:08:25

well until tonight i’ve had to re-run boot dev for every change

thedavidmeister14:08:50

so even being one behind is an upgrade >.<

onetom14:08:23

until you waste an hour because of it, then you just negated that gain 😜

thedavidmeister14:08:30

well even saving 10s here and there helps my sanity

onetom14:08:33

but then again, i don't have a better recommendation than using boot-system which would require a serious rehaul of the app using https://github.com/tolitius/mount might be better but im yet to figure out how can it form a coherent flow with intellij....

onetom14:08:06

also my recommendation is to get the fastest possible machine you can possibly purchase with at least 16GB ram

thedavidmeister14:08:11

i’m just using atom

onetom14:08:15

it should pay back fast...

thedavidmeister14:08:30

i’m not actually making any money off my app yet, so...

thedavidmeister14:08:44

if i get some customers one day i will 🙂

onetom14:08:09

you mean u don't have any other source of income for an upgrade?

thedavidmeister14:08:51

oh, i literally just moved house

thedavidmeister14:08:00

and have a huge backlog of shit to buy for the apartment

onetom14:08:15

like a faster machine? ... ;-/

thedavidmeister14:08:20

haha, somewhere on the list

thedavidmeister14:08:25

i’m going to start with “kitchen table"

thedavidmeister14:08:34

before “another 8gb of ram"

onetom14:08:39

fuck the curtains and carpet...

onetom14:08:42

that can wait

dm314:08:55

skewed priorities... 🙂

thedavidmeister14:08:05

lol, my gf is getting an upgrade before me

thedavidmeister14:08:19

can you imagine, she’s trying to compile this stuff on an air

thedavidmeister14:08:25

it just doesn’t really happen

onetom14:08:37

see you can get a kitchen table with 1GB ram first, so you get another 8GB into your computer instead and only then upgrade your kitchen table 😉

onetom14:08:04

maybe you should upgrade your girlfriend instead 😉

thedavidmeister14:08:24

well we had an imac at the last place, that was ok

thedavidmeister14:08:29

but there’s just nowhere to put it here

thedavidmeister14:08:46

we’ll get there

onetom14:08:17

aw... 🙂

thedavidmeister14:08:49

getting late here

thedavidmeister14:08:31

thanks again for the help