Fork me on GitHub
#boot
<
2016-07-29
>
micha00:07:08

also you can use boot --checkouts foo.bar/baz:1.2.3-rc7 repl

micha00:07:25

that way you don't need to molest your build.boot to shim in a checkout dep

micha00:07:51

@martinklepsch: i'm not going to be able to make it to Europe for clojutre or euroclojure 😕

danielsz00:07:35

Hello everybody, a short note to users of system to say that the latest snapshot version will act as a tools.namespace wrapper if a system is not supplied. The use case was brought up on Stack Overflow. http://stackoverflow.com/a/38625722/465458

michael.heuberger02:07:29

hello folks - having a boot-cljs performance problem here and wonder if one of you can help? a work colleague of mine and me, we share exactly the same source code but on his machine the boot task to reload after changes is 20 times faster which is weird

michael.heuberger02:07:59

we both have the same environment and our mac os xmachines are have very close performance characteristics

michael.heuberger02:07:20

our boot cljs task starts like this

michael.heuberger02:07:22

(deftask development []
  (task-options! cljs {:optimizations :simple :source-map true}
    reload {:on-jsload ‘abc-ui.app/init})
  identity)

michael.heuberger02:07:11

and here the stats:

michael.heuberger02:07:12

Adding :require [devtools.core dirac.runtime] and :init-fns [devtools.core/install! dirac.runtime/install!] to app.cljs.edn...
Adding :require adzerk.boot-reload.init1173 to app.cljs.edn...
Compiling ClojureScript...
• js/app.js
Writing target dir(s)...
Elapsed time: 22.480 sec

michael.heuberger02:07:37

whereas his are below 2 sec

micha03:07:22

@michael.heuberger: do you have a ~/.boot/profile.boot?

micha03:07:34

also maybe JVM options?

micha03:07:48

are you running the same JVM version?

micha03:07:13

just thinking of things that could make a difference other than code in the project

micha03:07:34

also maybe try du -sh ~/.boot/cache, like maybe there is an insane amount of stuff cached by boot?

micha03:07:56

and you can turn on verbose output with boot -vv ... to see what it's doing that takes so long

michael.heuberger04:07:27

@micha: thanks for that - no i dont have a ~/.boot/profile.boot … should i have one?

michael.heuberger04:07:53

our jvm options are BOOT_JVM_OPTIONS='-Xmx2g -client -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -Xverify:none XX:MaxMetaspaceSize=100m’ and we both share that together

micha04:07:18

no, i was just trying to think of things that could affect what boot does other than the project itself, which is the same for both of you

michael.heuberger04:07:31

~/.boot ❯❯❯ du -sh ~/.boot/cache
228M	/Users/michael-heuberger/.boot/cache

micha04:07:39

not at all

micha04:07:19

there is a clojurescript compiler option you can use to see where the compiler is spending its time

micha04:07:25

(cljs :compiler-options {:compiler-stats true ...

michael.heuberger04:07:31

and are the jvm options good?

micha04:07:02

the jvm options look fine, especially if they're the same for both setups

michael.heuberger04:07:14

tried -vv just now and i can see that when i change a single file, it registers all files and fires lots of events

micha04:07:39

yeah that's normal, because the cljs compiler can end up changing a lot of files

michael.heuberger05:07:12

but why does it also re-copy svg files when i only have changed one cljs file?

micha05:07:38

it shouldn't be doing that

micha05:07:09

what does it print that indicates that it is?

micha05:07:19

are you using the target task?

michael.heuberger05:07:28

Filesystem: copying index.html... Filesystem: copying vectors/spark/mail_search_24px.svg... Filesystem: copying vectors/spark/logo_white.svg... Filesystem: copying vectors/spark/logo_soft_white.svg... Sync complete. Released java.util.concurrent.Semaphore@7161d8d1[Permits = 1]... Elapsed time: 21.023 sec `

michael.heuberger05:07:01

yes, using target task

micha05:07:12

you could leave that off

micha05:07:24

it won't have to copy as much

micha05:07:42

it can be more clever with things that don't escape boot's internal temp dirs

micha05:07:51

like use caching and hard links and so on

michael.heuberger05:07:06

an idea for a future version?

micha05:07:18

it already does that! 🙂

michael.heuberger05:07:14

still slow despite of removing the target task

micha05:07:23

maybe it's hardware errors

micha05:07:42

if a disk is getting rickety it will get very slow to write to it

michael.heuberger05:07:43

huh? i have a new mac book

micha05:07:51

and you won't necessarily see exceptions

micha05:07:15

and both machines have the same version of OSX?

micha05:07:23

the fast one and the slow one?

micha05:07:10

so what could be different?

micha05:07:16

i don't get it

michael.heuberger05:07:18

that’s what i am asking myself too

michael.heuberger05:07:25

how can i narrow down the problem here?

micha05:07:50

i guess if you can narrow down which tasks cause the problem

micha05:07:06

remove tasks until it gets equally fast

michael.heuberger05:07:15

when i set optimizations to none, then it is very fast

michael.heuberger05:07:20

but not for simple

micha05:07:31

interesting observation

michael.heuberger05:07:53

must be related to the simple optimization routine

micha05:07:03

and your buddy is using :simple too, but is faster, right?

micha05:07:18

haha ok just making sure

michael.heuberger05:07:25

if we both use none, then we are both equally fast

micha05:07:33

memory pressure

micha05:07:49

google closure needs a lot of memory to perform optimizations

micha05:07:02

if you put pressure on it it will still work, but very very slowly

micha05:07:21

are you sure the memory situation is similar in both cases?

michael.heuberger05:07:34

reason we switched to simple optimizations is because source maps wont work when optimizations are set to none

michael.heuberger05:07:43

for local development

michael.heuberger05:07:51

i ll check with him re memory

micha05:07:58

source maps should work

micha05:07:07

i use them all the time

micha05:07:16

are you using boot-cljs?

michael.heuberger05:07:17

no, the do not work when having no optimizations

michael.heuberger05:07:23

yes, boot cljs and dirac

micha05:07:40

they do work, there is something else wrong with your setup

micha05:07:52

i use source maps with :optimizations :none all the time

micha05:07:05

i mean it's the default

michael.heuberger05:07:10

hmm, weird - why does it work for me only when optimized with simple?

micha05:07:33

are you setting :compiler-options?

micha05:07:50

boot cljs task does a lot of work to get source maps to work

michael.heuberger05:07:17

no, we do not use any compiler options at all

micha05:07:22

but if you go in and set compiler options then it can't help, because a lot of compiler options are related to each other in weird ways

micha05:07:52

so source maps should definitely work with :none

micha05:07:16

you can see that by doing this:

micha05:07:41

mkdir src ; echo '(ns foo) (println "hi")' > src/foo.cljs ; boot -BP -d adzerk/boot-cljs -s src cljs target

micha05:07:10

you will see the source maps in the target dir

micha05:07:34

and you can make a html file that loads the main.js file and you'll see the source maps in the js console

michael.heuberger05:07:28

~/C/test ❯❯❯ mkdir src ; echo '(ns foo) (println "hi")' > src/foo.cljs ; boot -BP -d adzerk/boot-cljs -s src cljs target Downloading https://github.com/boot-clj/boot/releases/download/show/boot.jar... Exception in thread "main" java.io.FileNotFoundException: https://github.com/boot-clj/boot/releases/download/show/boot.jar at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1836) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) at java.net.URL.openStream(URL.java:1045) at Boot.download(Boot.java:173) at Boot.install(Boot.java:224) at Boot.main(Boot.java:243)

micha05:07:58

try doing that in like /tmp or something

michael.heuberger05:07:14

why should this make a difference?

micha05:07:55

target/index.html

<!doctype html>
<html>
  <head>
    <script src="main.js"></script>
  </head>
  <body>
    hi
  </body>
</html>

micha05:07:59

that's the html file i used

micha05:07:16

well yeah, there's no boot.jar there

michael.heuberger05:07:22

can we make it easier?

michael.heuberger05:07:35

i ll pull an example from github and go from there?

michael.heuberger05:07:42

which one you recommend?

micha05:07:10

boot thinks you want boot version "show"

micha05:07:17

but "show" isn't a version of boot

micha05:07:48

where you see "show" in the url there you should be seeing like "2.6.0" for example

michael.heuberger05:07:18

dont see what’s wrong

micha05:07:37

doesn't the boot-cljs README have examples?

micha05:07:54

my internet is so bad i can't even load anything

micha05:07:06

or maybe github is down

micha05:07:50

my internet is slower then a 56k modem

michael.heuberger05:07:50

but thanks for your feedback

michael.heuberger05:07:57

maybe someone will have an idea next week?

micha05:07:10

did you try in an empty dir?

micha05:07:30

cause maybe you have a corrupted boot.properties file or something

micha05:07:36

in the current directory

micha05:07:49

or ~/.boot/boot.properties

micha05:07:02

something is setting BOOT_VERSION=show

micha05:07:04

somewhere

micha05:07:28

and that's nonsensical

jetzajac13:07:55

Hello guys. I’m pretty new to boot, just thinking about migrating from lein. I see a boot as a good replacement of out current project build-system because we have multiple lein projects that depend on artifacts of each other. I want to have a single boot file to build them all in a right order. but how can I “narrow down” to some directory with updated env? subprojects have different deps, source-paths etc. Is it even a good idea to use boot in this case? how do you build different artifacts and add them to dependencies of other ones?

micha15:07:32

@jetzajac: i would still recommend making separate versioned artifacts for the separate projects

micha15:07:48

meaning individual pom.xml and jar and install in maven

micha15:07:06

you can use the --checkouts option to boot to get live updating

micha15:07:18

or :checkouts in set-env!

micha15:07:28

so the way i would do it is like this

micha15:07:37

suppose i have projects A, B and C

micha15:07:30

and i'm working on C that depends on A and B

micha15:07:39

and maybe i want to also hack on A at the same time

micha15:07:53

so i'd start incremental compilation in project A

micha15:07:02

boot watch pom jar install or something like that

micha15:07:24

that will rebuild the A project jar continuously and install it in local maven

micha15:07:55

then in project C i would do like boot --checkouts A:1.2.3 watch build

micha15:07:08

assuming 1.2.3 is the version of A that i'm working on

micha15:07:21

and building with the boot watch ... above

micha15:07:47

this way the projects are completely separate and there isn't a need to merge the build process

micha15:07:02

because boot checkouts consume the jar, not the source files

micha15:07:39

decouples the builds, in case there is some stuff you must do to build A and/or package it

micha15:07:51

is that what you're looking for?

jetzajac15:07:44

perfect, I got you. But what if in order to run C i have to run A and B first? (A and B are microservices, not libs)

micha15:07:05

you can just start them first 🙂

jetzajac15:07:23

but what about magical (boot (dev)) ?

micha15:07:30

you can use boot-in-boot also, if you want to

micha15:07:33

then you'd have a single boot file

jetzajac15:07:07

are there any examples of boot-in-boot you know?

micha15:07:33

that's one example

jetzajac15:07:47

awesome! that’s what i was looking for

micha15:07:52

also in boot itself @richiardiandrea developed some great tooling around parallel multi-module builds

micha15:07:09

but the repo i linked might be a good place to start

micha15:07:18

cause it's sort of deconstructed

richiardiandrea15:07:27

hey the repo looks cool, I'll have a look too

micha15:07:01

the ont thing the repo demonstrates that you're interested in is using semaphores to synchronize around dependencies

micha15:07:12

those example modules have dependencies on each other

jetzajac15:07:21

thank you guys. I see boot is so powerful that it is hard to imagine what is possible and what is difficult to achieve with it

micha15:07:38

yeah all boot does is get you to the point where you can program in clojure

micha15:07:43

it "bootstraps" you

micha15:07:49

then you can do anything

micha15:07:14

the downside is that you can't do anything without programming though 🙂

jetzajac15:07:15

I dream of a single repl to rule the dev, build, deploy processes of the whole proj

micha15:07:31

yes everything you can do in boot can be done from the repl

micha15:07:51

the command line is just a thin facade over the repl anyway

micha15:07:13

well over evaluation of clojure code

micha15:07:26

but yeah, you can totally just live in the repl

micha15:07:50

like if you'd do boot build --option 123 at the command line

micha15:07:02

you can do (boot (build :option 123)) in the repl

jetzajac15:07:48

yeah this “homomorphism” is very natural and handful too. CI, build script, IDE are all the same.

micha15:07:17

have fun! if you get stuck we can help you here 🙂

micha15:07:07

powerlaces, awesome name

richiardiandrea15:07:24

now I am spamming

dominicm15:07:36

I call dibs on unicornlaces, for when I need them.

dominicm15:07:12

Oh, Nike's laces were called Power Laces. Unicorn Laces seem less awesome now

micha15:07:21

Dr. Scholl's Odor Eaters

micha15:07:30

less smelly boots

dominicm15:07:59

Holes: Mr. Yelnats created Sploosh for foot deodorizing.

alandipert15:07:03

sploosh, definitely a good name for something

micha15:07:11

haha "foot deoderizing"

flyboarder17:07:12

@dominicm: unicornlaces 👍

juhoteperi17:07:23

@richiardiandrea: I'm still wondering what is the benefit of boot-cljs-devtools? Does it help with Dirac?

richiardiandrea17:07:13

@juhoteperi: yes it basically adds both cljs-devtools and dirac at the same time

juhoteperi17:07:19

If only using cljs-devtools, looks like it needs more lines to enable than just cljs-devtools alone

richiardiandrea17:07:29

in lambone I have a switch to choose either one or the other

juhoteperi17:07:25

Also, does it run devtools init code each time cljs code is reloaded or only during initial load? I found it better to only run devtools init once

juhoteperi17:07:53

Hmm. Never used Dirac but looks like the dirac option code in cljs-devtools is useful

richiardiandrea17:07:51

that's a good point I think now I have it inside a watch but as you say it might be better outside...

juhoteperi17:07:16

Also, it depends on if you are using init-fns or just require (or preloads)

richiardiandrea17:07:34

it is using init-fns atm

richiardiandrea17:07:52

preloads was not out when it was originally done..

juhoteperi17:07:58

IIRC init-fns are ran each time

richiardiandrea17:07:43

yeah that will change in the future...i inherited this project so that is something to do

richiardiandrea17:07:17

also @juhoteperi a question, do I need to put the boot-cljs-repl task in a watch ? because that is another thing that it's better to do once only...

juhoteperi17:07:41

@richiardiandrea: you can leave it inside watch, it has logic to run only once if no changes in cljs compiler opts

kenny18:07:27

How does :checkouts work with the REPL? If I add a dep to my checkouts vector and run boot watch pom jar install in that dep that I added to my checkouts vector, it seems the changes don't get pushed to the REPL. Is there something I need to do to make this work?

richiardiandrea19:07:32

random question: is a pod aware of environment variables? my instinct says no, just want to double check

luke19:07:37

Is there any way to cause boot to add multiple resource directories as separate directories on the classpath? Currently it looks like its merging them into one FileSet (the "mutable classpath"). But this isn't great for loading files that there are supposed to be multiple copies of (e.g, data_readers.clj). For example, say I have a boot environment with `:resource-paths #{"source" and "../other-project/src"}. Even though both "src" directories have a data_readers.clj file, I can only see one of them since they're getting merged into a single temp directory. Is there any way to avoid this?

martinklepsch19:07:48

@luke: I don't think there's a way around this. Although if you're loading "other-project" I assume you're also depending on the jar of that project which could import the data_readers.clj?

luke19:07:04

That’s true, I could make sure the jar was also on the classpath

luke19:07:18

but would the source dir take precedence over overlapping files? I guess it would, wouldn’t it?

martinklepsch19:07:19

@richiardiandrea: I think System/getenv should work in pods just as usual

richiardiandrea19:07:09

mmm...need to try that, I had a problem with dirac env vars in the repl task

martinklepsch20:07:01

@luke: precedence in which way? I don't know much about data_readers.clj — is Clojure going to merge multiple of them in some way?

luke20:07:16

clojure knows how to merge them if it sees multiple files, yes

luke20:07:24

but boot only lets you see one file

luke20:07:49

by “precedence” I mean having the same file in a source dir and also in a jar, and then requiring it

martinklepsch20:07:34

ah! I'd assume source-dir takes precedence, not sure if that's correct but I think that's essentially how checkouts work

micha21:07:59

@kenny: once a dependency is loaded you can't make it a checkout

micha21:07:11

because once a jar is added to a classloader there is no way to remove it

micha21:07:34

so what you need to do is ensure that the --checkouts option to boot is used

micha21:07:58

or have :checkouts in your build.boot when you add the dependencies

micha21:07:24

i recommend using boot -c foo.bar/baz:1.2.3 repl

micha21:07:33

for example

kenny21:07:24

Not sure how that is different than what I said

micha21:07:03

did you do boot --checkouts ... repl?

kenny21:07:21

Well, I had it in :checkouts so yes.

micha21:07:06

if you don't see the thing updating it's because the dependency was loaded before boot was told that it's a checkout dep

micha21:07:11

and then it's too late

micha21:07:43

using the command line option ensures that that won't happen

kenny21:07:48

When I start the REPL it would say "Adding checkout dependency ..."

kenny21:07:52

Then start the REPL

kenny21:07:03

I'll try using the command line option

kenny21:07:57

Yup, same deal -- didn't reload in the REPL

micha21:07:14

do you have an example?

micha21:07:24

i guess it's a pain to make an example

kenny21:07:51

I can make one and send the parent folder as a zip

kenny21:07:12

PMed you the zip

kenny21:07:42

Run boot auto-build in the child and boot -c child:0.1.0-SNAPSHOT repl in the parent.

kenny21:07:24

Oops.. Comment out second-fn first 🙂

micha21:07:21

it's working for me

micha21:07:45

did you do (require '[child.core :as c] :reload) after changing the child code?

kenny21:07:44

I did not. I'll try

micha21:07:02

ah yeah, you'd have a cached namespace in the repl without the :reload bit

kenny21:07:34

Works 🙂

kenny21:07:15

Hmm.. Wonder if Cursive has an easy way to do that...

kenny21:07:35

Thanks for the help 😄