Fork me on GitHub
#boot
<
2019-05-10
>
flyboarder02:05:14

@jeroenvandijk there are some errors where boot internally (like catch java errors) that clojure 1.10 now returns as nil

jeroenvandijk07:05:30

Thank you. One of my tests was failing due that nil value. I just changed the code to make it pass, but I couldn't find where it was coming from. I guess it is during the eval-ing of the repl. Although that would not explain why it was failing during my (custom) boot test task. Ah well

jeroenvandijk14:05:26

I'm having a problem with the delay in the update of boot fileset again. An external file watcher process (hawk) is too fast compared to boot's fileset updates. Is there a way to use boot's watch task to run an arbitrary function?

dave14:05:27

you could turn your function into a task and include it in the pipeline after the watch task

dave14:05:34

i.e. boot watch do-my-thing

jeroenvandijk14:05:08

do you know how I could do it from the repl? I'm a bit struggling with the function composition

jeroenvandijk14:05:34

something like this (watch (my-other-task)) ?

dave14:05:40

(boot (watch) (my-other-task))

dave14:05:01

each task is a function that returns a function

dave14:05:17

so that they can compose together in a pipeline

jeroenvandijk14:05:17

Is there a way to cancel the watch process?

jeroenvandijk14:05:47

Maybe not very clean but this seems to work:

(def x (future (boot (watch :verbose true) (fn [& args] (println "fooo" args)))))
(future-cancel x)

dave14:05:42

i've seen it done that way

jeroenvandijk14:05:13

This is pretty cool actually 🙂

jeroenvandijk14:05:53

I also see why my previous (Thread/sleep 25) didn't work. For this project a fileset update takes around 90ms

jeroenvandijk15:05:20

[Ignorant question] Is that a hard problem to solve?

dave15:05:03

this reminds me of a post @alandipert wrote on the adzerk blog a couple years ago: https://adzerk.com/blog/faster-clojure-development/

jeroenvandijk15:05:07

I think it's actually more than 90ms as that's what the watcher process reports, but then there is also the picking up of the change it self (through polling?)

jeroenvandijk15:05:26

ah cool i'll have a look. Thanks @dave

jeroenvandijk15:05:23

Compared to my alternative implementation, independent of boot, with hawk (https://github.com/wkf/hawk) This is a significant slow down

jeroenvandijk15:05:41

hmmm ok i think i can see why it is slow

jeroenvandijk15:05:29

I'm guessing that could be made a lot more efficient with a different approach like hawk

jeroenvandijk15:05:43

@flyboarder Do you have opinions on the above? I could try to make this a bit faster

jeroenvandijk15:05:39

hmmm maybe i don't understand how it works. I'll look into it a bit more before i say stupid things

jeroenvandijk15:05:19

This is how I understand it: There is a native (fast process) implemented in boot.watcher. However due to the LinkedBlockingQueue there is a default polling latency of 10ms (https://github.com/boot-clj/boot/blob/master/boot/core/src/boot/core.clj#L749). This latency is for any project. On top of that and this hurts bigger projects is a function that runs a fingerprinting process of the whole tree (watcher, boot/core.clj#L751). This function goes over all files on every change https://github.com/boot-clj/boot/blob/7709941bcef20de55b940ec18885677df33b7557/boot/pod/src/boot/file.clj#L247-L248 This is unnecessary as the native process implemented in boot.watcher would be able to tell exactly what file has changed. I'm sure i'm missing something either in the problem space or in my above analysis

flyboarder16:05:35

@jeroenvandijk this is an area I want to improve for sure! There is work towards a cleaned up version of the fileset (fileset branch on github) which I feel will help simplify the watch task

flyboarder16:05:09

But I agree, my large projects are now taking a minute to build even tho the compilation step is less than a second

flyboarder16:05:45

I also have to wait a while before changes are picked up and a recompile is triggered

flyboarder16:05:59

So I think our current approach is flawed

flyboarder16:05:36

Actually looking at the watcher, it should probably be rewritten anyway, it falls under the remove :boot/from code goal for v3