Fork me on GitHub
#boot
<
2016-06-29
>
currentoor00:06:15

got it, thanks

currentoor00:06:20

i'll try booting

micha00:06:04

yeah it looks like your problem is not with cljx

micha00:06:36

you don't need the cljx task because it's already transpiled and whatnot in the jar

seancorfield04:06:05

Just updated to the master of Emacs Live and the Boot support has gotten much, much better: it now runs boot … repl -s wait with the correct dependencies and middleware for CIDER and refactor-nrepl automatically! No more hacky Boot file stuff to add all of that!

seancorfield04:06:37

Unfortunately there’s a tiny bug in the CIDER conf.el file that prevents cljr-refactor from loading (I submitted a PR).

onetom04:06:07

i've managed to peek into the dependencies of a pod for the 1st time!

boot -d tailrecursion/boot-static:0.0.1-SNAPSHOT serve -p 8000 show -P boot-static -d

onetom04:06:28

in retrospect it's understandable but it took a while until i really understood this is what i have to do... i wish i had this being explained to me earlier on. but where would i expect to see this documented?

onetom04:06:18

of course micha and the existing documentation did a great job explaining the different bits and pieces of all this but it was still non-obvious to combine all this

seancorfield04:06:06

Ya, sometimes Boot is almost too flexible and it’s hard to know where to go to read up on a problem you’re trying to solve! 🙂

onetom04:06:34

i haven't found explanations about the following for example: 1. the -d some/dependency:1.1.1 option is doing an implicit (require '[dep.ns :refer :all]) 2. to have boot show -l access pods of other tasks i should actually call the task even if im not interested in it's side-effect

kgzm04:06:25

I've seen it said that one can pull in a dependency with boot during a repl session without having to restart the session, how does that work?

seancorfield04:06:30

@onetom: Not sure what you mean by 1. there?

seancorfield04:06:46

Re: 2. Yes, since show -l shows "active pods" you would need to run a task to get its pods up and running (and active).

biscuitpants08:06:14

can someone explain something that i read on the boot-clj site? this part:

biscuitpants08:06:22

> artifacts can never be stale. there is no clean task

biscuitpants08:06:40

not sure i understand how / why that happens in boot, but not in leiningen?

martinklepsch08:06:32

@kgzm: you can call boot.core/set-env! inside a repl

martinklepsch08:06:19

@biscuitpants: leinigen writes to target potentially leaving files from previous builds

martinklepsch08:06:53

In some cases these files are also used for incremental compilation etc.

martinklepsch08:06:56

In boot the state is fully represented through the fileset and so this problem can easily be avoided by detecting which files are missing from the fileset and deleting them

martinklepsch08:06:20

(There are probably more details to this but I hope this gives an idea)

biscuitpants08:06:08

ah i see, thank you @martinklepsch that makes a lot of sense

lwhorton13:06:04

if I were to run a task such as npm install inside a dosh, where a bunch of files are added — how do I get those files added into the fileset?

lwhorton13:06:27

rerunning core/input-files fileset just provides the set without the newly created files… do I need to somehow npm install into the result of the tmp-dir!, in which case they will be detected by boot? can I manually add them somehow with standard fileio? such as:

;; move files into fileset
        (doseq [f files]
          (let [in-file (boot/tmp-file f)
                target (io/file tmp (boot/tmp-path f))]
            (io/make-parents target)
            (io/copy in-file target)))

micha13:06:54

the usual way to do this @lwhorton is like this:

lwhorton13:06:01

btw @micha I found a way to do npm inside boot (I think) by simply including a package.json as a resource, and running a dosh with bindings *sh-dir* set to wherever the package.json is declared.

micha13:06:05

(deftask foo
  []
  (let [tmp (tmp-dir!)]
    (with-pre-wrap [fs]
      (dosh "doit" "--output-dir" (.getPath tmp))
      (-> fs (add-resource tmp) commit!))))

lwhorton13:06:23

at least on the first run through all the files show up, all the installed stuff can be touched via normal npm operations

lwhorton13:06:51

it’s after a file changes things bork because they cannot find anything inside node_modules — i.e. it’s not in the fileset (I think)

micha13:06:24

seems like you probably want to separate npm stuff from your boot stuff

micha13:06:36

because npm is stateful

micha13:06:34

the thing i pasted above will get the node stuff into the fileset

micha13:06:22

you can also bind boot.util/*sh-dir* to the temp dir path when you call dosh

micha13:06:33

then npm will presumably create node_modules in there

lwhorton13:06:48

i see, i was kind of hacking the system, changing sh-dir to get the tmp path to the file:

(defn npm-path []
  (.getPath (.getParentFile (io/file (io/resource "lwhorton/boot_stylus/package.json")))))

lwhorton13:06:16

and then running with:

(binding [u/*sh-dir* (npm-path)]
          (do
            (u/dosh "npm" "install”)

lwhorton13:06:23

but that didn’t seem to actually put the items into the fileset

micha13:06:27

(binding [boot.util/*sh-dir* (.getPath tmp)]
  (dosh "npm" "install"))

micha13:06:50

you need to add them and then call commit! as in the example above

lwhorton13:06:56

i see.. i should be able to manually put the package.json into the fileset at tmp, then do the above to install everything else (using package.json) into tmp

micha13:06:17

(deftask foo
  []
  (let [tmp (tmp-dir!)]
    (spit (io/file tmp "package.json") package-json)
    (with-pre-wrap [fs]
      (binding [*sh-dir* (.getPath tmp)]
        (dosh "npm" "install"))
      (-> fs (add-resource tmp) commit!))))

lwhorton13:06:19

very helpful, cool

lwhorton13:06:33

let’s see how this goes..

lwhorton14:06:18

suweet… worked wonderfully micha

lwhorton14:06:10

now the task is a ‘self contained’ npm environment where the only external dependency is node.. tasks that need modules install them on the fly, and cache them so re-run is quick

micha14:06:38

wow awesome!

micha14:06:55

are you using cache-dir! for the caching?

micha14:06:11

or by re-run do you mean incremental compile?

lwhorton14:06:25

incremental compile, not cache-dir though that’s probably a smart move

micha14:06:44

the only issue there is that you need to know when you invalidate the cache then

lwhorton14:06:45

i’m using cache-dir to remember the css-modules hash, but just a clojure’d atom installed (atom false) * to remember if i need to install or not

micha14:06:26

sounds pretty legit to me

esp117:06:59

i’m making some standalone boot scripts as in: https://github.com/boot-clj/boot/wiki/Scripts. is there a way to get at the path of the script? i want to be able to execute stuff from the same dir as the script, even if the script is executed from another directory

esp117:06:40

e.g. some/dir/path/script.boot instead of ./script.boot

micha17:06:45

@esp1: yes that should be bound to a dynamic var in the boot.core namespace

micha17:06:24

@esp1: bot.core/*boot-script*

esp117:06:32

perfect, thanks!

richiardiandrea17:06:55

awesome did not know that

esp119:06:28

did something happen to boot’s uberjar performance? i just cloned and ran https://github.com/micha/boot-uberjar-perf and this is what i got:

15:27 $ time lein uberjar
Compiling foo.bar
Created /Users/esp/Code/contrib/boot-uberjar-perf/target/bar-0.1.0-SNAPSHOT.jar
Created /Users/esp/Code/contrib/boot-uberjar-perf/target/bar-0.1.0-SNAPSHOT-standalone.jar

real	0m7.515s
user	0m7.670s
sys	0m1.841s
✔ ~/Code/contrib/boot-uberjar-perf [master|✔]
15:32 $ time boot uberjar
Adding uberjar entries...
Compiling 1/1 foo.bar...
Writing pom.xml and pom.properties...
Writing project.jar...
Sifting output files...
Writing target dir(s)...

real	0m47.499s
user	0m10.615s
sys	0m5.352s

esp119:06:45

47.5 seconds for boot vs. 7.5 seconds for lein

juhoteperi19:06:31

@esp1: Which Boot version this is? I think there was improvements in some version

esp119:06:12

it’s referencing BOOT_VERSION=2.5.0-SNAPSHOT in the boot.properties file for that project. i’ll see if it makes any difference to change that to 2.6.0

juhoteperi19:06:49

Also, does your target dir contain only the uberjar or all the files?

esp119:06:58

only the jar

esp119:06:18

other stuff is elided by sift

juhoteperi19:06:19

Okay, then the difference wont be caused by writing the other files to disk

esp119:06:08

same result with BOOT_CLOJURE_VERSION=1.8.0 and BOOT_VERSION=2.6.0

juhoteperi19:06:38

Perhaps I remember incorrectly and it was not fixed yet

esp119:06:14

also the readme for https://github.com/micha/boot-uberjar-perf shows similar timings for lein and boot

esp119:06:06

i just can’t recreate that result...

micha20:06:41

hm i'll take a look

micha20:06:33

@esp1: this is what i get, changing boot.properties to point to the latest versions of boot and clojure:

micha20:06:07

@esp1: try running boot -P uberjar for your test, please

esp120:06:59

ok, running..

micha20:06:35

that will disable evaluation of boot.properties

micha20:06:47

in case something in there is interfering

micha20:06:02

with the phenominal PERFz

esp120:06:39

ok that took 59 secs, but i had another process running so let me wait until that finishes and i’ll try again. i may try rebooting my machine in a bit to see if that helps

micha20:06:05

what kind of a machine is it?

micha20:06:10

and which os?

esp120:06:16

macbook pro core i7 16 gb ram

micha20:06:37

with el captain?

esp120:06:51

yup 10.11.5

micha20:06:57

maybe something in osx is being slow

micha20:06:10

i only have linux here

esp120:06:21

it’s running fast for u?

esp120:06:40

16:08 $ time boot -P uberjar
Adding uberjar entries...
Compiling 1/1 foo.bar...
Writing pom.xml and pom.properties...
Writing project.jar...
Sifting output files...
Writing target dir(s)...

real	0m50.957s
user	0m10.430s
sys	0m5.244s

micha20:06:03

yeah i have a thinkpad x250 with 16G memory and i7 5600u processor

micha20:06:22

takes like 6s here

esp120:06:25

let me try rebooting.. back in a min

micha20:06:30

also i have an ssd drive

esp120:06:40

yeah i’m on ssd too..

juhoteperi20:06:09

btw. micha, I finally ordered t460s to replace x220, I had enough of the 1366x768 display 😄

micha20:06:27

man the x220 is so nice though

micha20:06:44

but yeah i have been also looking at those

micha20:06:09

the asymmetric keyboard ararngement is stopping me from buying one 🙂

juhoteperi20:06:05

Yeah x220 keyboard is super but as I doubt there will ever be new laptops with same kind of keyboards I decided I have to adapt to new keyboard sometime

micha20:06:28

yeah it's a shame

esp120:06:13

i’m back - reboot didn’t help much. i’m getting 54.5 secs now.

esp120:06:27

most of the time is taken writing project.jar

esp120:06:34

if it’s fast for you i guess we can chalk it up to an osx filesystem thing, altho lein runs much faster (8 secs just now)

micha20:06:50

yeah that's what boot should be doing too

micha20:06:08

what if you remove the jar task from the pipeline?

micha20:06:11

just to see

micha20:06:30

it will still be doing all the uberizing

esp120:06:47

11.9 secs

micha20:06:52

$ boot uber aot pom sift target

esp120:06:04

so yeah, much better

micha20:06:34

hm it's 4.66s for me

esp120:06:49

i ran it again and got 9 secs

esp120:06:40

i did change the boot.properties to use clojure 1.8 and boot 2.6, dunno if that makes any difference

micha20:06:51

yeah i did that too

richiardiandrea21:06:22

ups we am thrashing the channel 🙂

micha21:06:02

@richiardiandrea: so is it just threads that need to be killed?

richiardiandrea21:06:24

at the moment I am using the function in that PR

richiardiandrea21:06:41

core async stuff, and db related shutdowns

micha21:06:47

yeah it's not clear what all of the things are doing there

micha21:06:06

but i am assuming that the functions that are called, like the datomic shutdown one, are just stopping threads

richiardiandrea21:06:07

yes it is killing threads basically and ExecutorServices

micha21:06:48

track down all the threads, find their classloader perhaps

richiardiandrea21:06:51

get all the threads and kill them

micha21:06:20

could be a configurable timeout

richiardiandrea21:06:23

thanks, it looks easier than tracking stuff one by one...

micha21:06:38

like if the threads don't die on their own within like 60s or something

micha21:06:47

then the reaper terminates them with extreme prejudice

richiardiandrea21:06:57

in my case I am executing tests so every watch tick I can kill them all I guess

micha21:06:09

The Thread Exterminator™

richiardiandrea21:06:42

so it looks like a shutdown-agents would suffice...it would be great to understand why it is not

richiardiandrea21:06:55

because we are calling .close on the shim already

micha21:06:13

because those other thread pools are not clojure agents

micha21:06:30

that's my understanding

richiardiandrea21:06:36

but the bigger concern is the contextClassLoader for those threads will hold a reference to the shim's classloader, which will prevent any classes loaded in the shim from being unloaded.

richiardiandrea21:06:08

so the threads created in the shim hold references to the shim

micha21:06:39

if you start a non-daemon thread, then the pod can't be GC until that thread is stopped

micha21:06:49

like consider the case where you start a webserver in production

richiardiandrea21:06:11

(I was talking about this problem in #C085AR8RE this morning)

micha21:06:44

i think we can inspect running threads with the MX bean stuff

micha21:06:48

and do the business

micha21:06:09

i think the root of the problem is that threads are treated specially by the garbage collector

micha21:06:44

they can't be collected until they have stopped, even if nothing holds references to them

micha21:06:59

because they are running some service that needs to keep running

richiardiandrea21:06:39

I am working on the reaper 😉

seancorfield22:06:15

If I have a build.boot file in one folder and for "reasons" I want to have (effectively) the identical build.boot file in another folder, is there a way to have a sort of "shim" Boot file in one place that just sort of "includes" the other one?

seancorfield22:06:28

Maybe some sort of load script code perhaps?

micha22:06:09

@seancorfield: sure, you can use clojure.core/load-file in one of them

seancorfield22:06:30

Cool. Glad it’s that simple.

micha22:06:52

incidentally that's what you can use to reload the build.bootin the repl

micha22:06:05

(load-file "build.boot")

richiardiandrea23:06:27

No I cannot come up with a solid solution..