Fork me on GitHub
#boot
<
2019-01-16
>
flyboarder01:01:24

@jeroenvandijk how stable has that been?

jaide03:01:35

Is there a boot noop to put into a pipeline when conditionally building it? Like (comp (when (= :env :dev) (draft))?

flyboarder03:01:20

@jayzawrotny ok so the boot task is like a double wrapped function

flyboarder03:01:00

when you create a task you are returning a function that returns a fileset

flyboarder03:01:20

so to conditionally wrap that you would use a cond->

flyboarder03:01:35

I do it often, one sec I have demos

jaide03:01:40

Ah right, the middleware so would need to be like (comp (if (= :env :prod) (draft) (noop)) where noop is like (fn [next-handler] (fn [fs] (next-handler fs))

flyboarder03:01:24

yeah I find the threading macro easy tho

(cond-> (identity)
  (= :env :dev) (comp (dev)))

flyboarder03:01:42

that way I can build up a pipeline of conditional tasks

flyboarder03:01:32

I could also see a really easy reduce-task that takes a coll of tasks and reduces them to a pipeline

jaide03:01:52

(comp
  (perun/global-metadata)
  (perun/markdown :md-exts {:all true})
  (perun/draft)
  (perun/print-meta)
  (perun/slug)
  (perun/ttr)
  (perun/word-count)
  (perun/build-date)
  (perun/gravatar :source-key :author-email :target-key :author-gravatar)
  (perun/render :renderer ')
  (perun/collection :renderer 'eccentric-j.site.index/render :page "index.html")
  (perun/tags :renderer 'eccentric-j.site.tags/render)
  (perun/paginate :renderer 'eccentric-j.site.paginate/render)
  (perun/static :renderer 'eccentric-j.site.about/render :page "about.html")
  (perun/sitemap)
  (perun/rss :description "Eccentric J's Blog")
  (perun/atom-feed)
  (sass)
  (cljs :optimizations (if (= build-env :production) :advanced :none))
  (clean :exclude #{#".git"})
  (target :no-clean true)
  (notify)))
Is my pipeline thus far, some of those steps are conditional depending on env and some of those steps may take different args depending on the env. Maybe I should manipulate a list then reduce it into a pipeline?

flyboarder03:01:15

I think the correct answer to your question tho is just return an identity fn as the task, so it returns the previous tasks fileset

jaide03:01:31

Otherwise I’m going to have (cond->) all over the place.

jaide03:01:49

Thanks, though you got me thinking I need to rethink my approach

flyboarder03:01:22

ok I see what you are doing, take a look at what I have done here: http://github.com/degree9/meta

jaide03:01:15

[meta] seems really nice! I may have just found a stack for clojure desktop app dev!

flyboarder03:01:09

yeah we are using it everyday

flyboarder03:01:53

those tasks use the thread approach, it seems you probably want to breakup the pipeline into smaller sub-tasks

jaide03:01:22

Ok, I was just starting to think that 🙂

jaide04:01:31

(deftask build
  "Build the blog source and output to target/public"
  [e build-env BUILD-ENV kw    "Environment keyword like :dev or :production"]
  (let [prod? (= env :prod)]
    (->> [(perun/global-metadata)
          (perun/markdown :md-exts {:all true})
          (when-not prod? (perun/draft))
          (perun/print-meta)
          (perun/slug)
          (perun/ttr)
          (perun/word-count)
          (perun/build-date)
          (perun/gravatar :source-key :author-email :target-key :author-gravatar)
          (perun/render :renderer ')
          (perun/collection :renderer 'eccentric-j.site.index/render :page "index.html")
          (perun/tags :renderer 'eccentric-j.site.tags/render)
          (perun/paginate :renderer 'eccentric-j.site.paginate/render)
          (perun/static :renderer 'eccentric-j.site.about/render :page "about.html")
          (perun/sitemap)
          (perun/rss :description "Eccentric J's Blog")
          (perun/atom-feed)
          (sass)
          (cljs :optimizations (if prod? :advanced :none))
          (clean :exclude #{#".git"})
          (target :no-clean true)
          (notify)]
         (keep identity)
         (apply comp))))

jaide04:01:28

Going this route, I think this should keep things pretty clean. I didn’t see a natural split in the steps because they all work towards building the blog. Without any of these steps I’m not sure the result would be meaningful.

flyboarder04:01:23

If it builds ™️

jaide17:01:49

Is there a way to get watch to respond to changes in a target/public directory instead of the resources dir?

micha23:01:01

the target dir is not immutable, it's the final IO at the end of the pipeline

jaide23:01:14

Sounds like I’m trying to use it incorrectly then.

micha23:01:55

one of the reasons why tasks like watch can be made simple and reliable is that they only manipulate immutable filesets

micha23:01:15

so you can "roll back" to a previous state easily

jaide23:01:22

The intention was to serve the files compiled to target/public as a website and when those files change run through boot-livereload. Sounds like I need to put those in the build pipeline.

micha23:01:13

you can use :asset-dirs or :resource-dirs

jaide23:01:00

Actually, let me back up. I’m learning boot while developing a static site with http://perun.io. I had a watch task that would build the markdown files, gen the html files, compile the scss and the cljs. But I’m curious if I could set it up to be a bit smarter. Currently if I change a markdown file both the scss and cljs get recompiled.

micha23:01:30

are you sure they're really getting recompiled?

micha23:01:43

so there are two usual approaches to build tooling

jaide23:01:47

Well the tasks are firing and output is being logged

micha23:01:53

you have an approach like Make

micha23:01:06

where it computes dependencies and triggers tasks that need to run

micha23:01:10

boot doesn't do it that way

micha23:01:19

boot always runs the pipeline every time

micha23:01:31

but each task can hold a reference to the previous immutable fileset

micha23:01:38

and it can efficiently diff the files

micha23:01:54

so each task is responsible for figuring out for itself whether it needs to do any work or not

micha23:01:09

so while those tasks will run, they probably are not actually doing any work

micha23:01:33

they diff the new and old filesets and see that no scss files changed, and just pass the cached results from the previous run onward

jaide23:01:49

I see, unfortunately they do slow down the recompile a bit. It’s not the end of the world though.

micha23:01:12

perhaps there is a reason why they need to recompile something?

micha23:01:24

or perhaps there is a bug in the task that can be found and fixed

jaide23:01:36

No wait, I see it

jaide23:01:39

You’re right

micha23:01:46

no line numbers there but see the fileset-diff part?

jaide23:01:54

Yeah I see it now

micha23:01:57

yep exactly

jaide23:01:09

Very cool!

micha23:01:31

the reasoning there was that only the task itself really knows what it depends on

micha23:01:35

it can get pretty complex

micha23:01:54

the build tool can't really know

jaide23:01:35

Good point. Also, I think I see the issue. It’s setup to diff the entire fileset which is why it’s always recompiling for me. Does this mean I could create another task to compose a sift with just .scss files with boot-sass to make it more efficient?

micha23:01:36

do you have a large number of scss files?

micha23:01:50

i know sometimes those things can have a ridiculous number of files

jaide23:01:14

Nope, so far just one but if I was using this build system for a bigger site it’s possible.

micha23:01:35

it's slow with only a small number of files?

micha23:01:43

the diffing is usually quite fast

micha23:01:04

since it's all immutable it can do various tricks

jaide23:01:50

I don’t think it’s the diffing that’s slow, I’m just saying if I change a markdown file: the fileset this task receives will be different than the previous fileset it diffs against and will recompile

micha23:01:20

it will only recompile if a scss file changes though

micha23:01:29

at least that's how it looks from the code

micha23:01:51

the (when (seq sources) ... part

jaide23:01:13

Right, and sources is the result of diffing filtered by the sass extensions

micha23:01:13

sources is the .scss files that changed

micha23:01:51

seems like it should skip recompiling if only a markdown file is changed, no?

jaide23:01:37

Agreed, boot-sass is definitely designed to work that way.

jaide23:01:08

I’ll try again tonight by removing previous steps from the build pipeline until it behaves as expected.

micha23:01:49

you can try boot -vv ... too

micha23:01:06

that will log a bunch of debug messages that might show something