Fork me on GitHub
#boot
<
2015-08-18
>
onetom01:08:15

Compiling ClojureScript...
• about-us/index.html.js
...
• videos/index.html.js
Retrieving ns-tracker-0.3.0.jar from 
Retrieving ns-tracker-0.3.0.jar from 
Retrieving clojurescript-1.7.48.jar from 
Retrieving google-closure-library-third-party-0.0-20150805-acd8b553.jar from 
Retrieving closure-compiler-v20150729.jar from 
Retrieving google-closure-library-0.0-20150805-acd8b553.jar from 
Retrieving closure-compiler-externs-v20150729.jar from 
Retrieving ns-tracker-0.3.0.jar from 
Retrieving closure-compiler-externs-v20150729.jar from 
Retrieving closure-compiler-v20150729.jar from 
Retrieving clojurescript-1.7.48.jar from 
...
Looks like it's not a good idea to download dependencies during runtime, especially not in parallel the same deps...

onetom02:08:38

When I looked into one of the boot-* "plugins" I saw it was also acquiring certain dependencies during runtime. While it's quite convenient from a development perspective, it's not very resilient during operation. I would prefer to gather all deps upfront, during the deployment process. In fact with NixOS I kinda have to do it anyway, because it won't allow my app to write just anywhere during runtime...

micha03:08:30

@onetom: you can do it, we do it for our docker images, for instance

micha03:08:30

it's not really possible to know for sure what dependencies you need until you run the program

micha03:08:42

like any program, you can't know what it will compute until you run it

onetom03:08:43

here is an example of the "problematic" approach im talking about: https://github.com/Deraen/boot-less/blob/master/src/deraen/boot_less.clj#L11

micha03:08:23

i suppose we do need synchronization around the maven machinery

onetom03:08:27

in this case u do know the exact dependency, no?

micha03:08:32

not really

micha03:08:37

it's part of a program

onetom03:08:42

imean in this boot-less case

onetom03:08:01

well, im trying to say that such logic should not be the part of the application whenever it's possible

micha03:08:31

did your maven repo get corrupted?

micha03:08:51

so that's good

micha03:08:04

there are ways to deal with the dependencies, like we do for docker

onetom03:08:21

it was just running on a t2.medium instance which has low network bandwidth so i too unnecessarily long download the same things 10+ times

micha03:08:14

you can prime the instance by running the task with a simple case, no parallel build

micha03:08:24

that will fetch all dependencies for you

onetom03:08:53

how can i turn off the parallel build? i missed the introduction of that feature

micha03:08:09

oh, you can just point it at one page

onetom03:08:58

err... how? simple_smile

micha03:08:03

boot could maybe be smarter about synchronization around deps

micha03:08:10

just make a task like this

onetom03:08:18

u mean just point the cljs task to 1 specific source?

onetom03:08:12

(deftask prime []
   (set-env! source-pathes #{src/one/page.cljs.hl})
   (comp (hoplon) (cljs)))

micha03:08:33

yep like that

micha03:08:45

we do things like this when we make our docker images

micha03:08:59

so the images contain all the maven deps already

onetom03:08:10

interesting idea... it would solve the long bootup time too to have something like this, since we often only need to work on 1 page...

onetom03:08:29

it's a bit hacky though, but will do for now

micha03:08:35

i wonder if we could make a boot task that adds a caching proxy in front of the maven stuff?

micha03:08:01

that would solve the craziness of parallel dependency resolution i think

onetom03:08:15

but then the proxy should handle parallel downloads of the same url efficiently

onetom03:08:48

however it should quite like supporting the "Single Unresponsibility Principle" 😉

micha03:08:42

haha i haven't heard of that

micha03:08:50

it sounds like something i would be into though

micha03:08:01

the story of my life, pretty much

onetom04:08:03

this was the most enlightening demonstration i've seen so far. the previous part also worth watching, tho that's a simpler example: http://cleancoders.com/episode/clean-code-episode-6-p1/show

danielsz13:08:03

alandipert: micha ping

danielsz13:08:46

alandipert: Hi!

danielsz13:08:09

alandipert: I have a working implementation of the gpg rewrite!

danielsz13:08:38

alandipert: Now I'm trying to do the integration with boot

danielsz13:08:17

alandipert: I'm having a difficulty when building boot.

danielsz13:08:07

alandipert: In boot/worker/project.clj I've updated the dep to [mvxcvi/clj-pgp "0.8.2"] but it doesn't seem to take.

danielsz13:08:55

alandipert: Is there anything I have to do additionally?

danielsz13:08:51

alandipert: I'm doing make deps and make install

micha13:08:45

@danielsz: the worker pod already has that available

micha13:08:06

it's already a dependency there

danielsz13:08:22

@micha it's an old version.

micha13:08:40

make clean, perhapd

micha13:08:47

s/perhaps

danielsz13:08:59

Sure, let me try

danielsz14:08:25

micha: OK, my bad.

micha14:08:46

all good?

danielsz14:08:29

micha: I'm so excited about this. I've removed a lot of code. It's much simpler now. I can't claim victory just yet because I'm still testing. Will report here soon.

micha14:08:46

!m danielsz

danielsz14:08:34

micha: OK, it compiles! Now I need to test with bootlaces. How do I build and install bootlaces locally? (boot pom install)? I forgot.

micha14:08:04

what are you using bootlaces for?

danielsz14:08:06

micha: bootlaces for push-jar

danielsz14:08:52

No, sorry: boot build-jar push-release

micha14:08:05

for boot itself?

danielsz14:08:56

With the new implementation, bootlaces doesn't need to check for credentials, so I did this change, but I forgot how to install my updated bootlaces in my local mvn repo.

micha14:08:11

oh i get it, sorry

micha14:08:25

boot build-jar builds and installs locally

danielsz14:08:32

Ah, that's it. Awesome

danielsz14:08:38

I've successfully deployed a release on clojars with the new implementation. Woohooo!

danielsz14:08:19

It's the best refactoring I ever did on a third-party project.

danielsz14:08:32

I hope you'll like it

danielsz14:08:14

Will issue a PR so you can review, test, comment, and instruct me further.

danielsz14:08:25

Sounds good?

alandipert14:08:31

neat, looking forward to checking it out!

micha14:08:30

sounds amazing!

danielsz14:08:21

I've pushed my changes. simple_smile

danielsz15:08:56

I'm dancing here at home!!!

danielsz15:08:11

dance of joy

danielsz15:08:41

Could you add me to adzerk-oss so I can push bootlaces changes. I forgot to fork it.

micha15:08:49

sure one sec

danielsz15:08:28

wow. that was an epic hacking session.

micha15:08:49

you should haz invite

danielsz15:08:03

OK, I've pushed the bootlaces changes as well.

danielsz15:08:36

All in all, there might be things that I overlooked, but this was a massive refactoring as far as I'm concerned, and I wanted to share as soon as possible.

danielsz15:08:03

At least I know it works, since I've pushed to clojars.

danielsz15:08:37

Please feel free to discuss.

tel15:08:11

What’s the preferred way to get boot on a CI server?

tel15:08:28

just download, chmod the script, and go?

danielsz15:08:38

@alandipert @micha the biggest joy is to remove code, simplify function signatures, and aggregate concerns as close as possible to the callee site.

martinklepsch21:08:13

@tel: in case your question isn’t answered yet: yes, download script, chmod, and potentially cache if possible.