Fork me on GitHub
#boot
<
2016-12-06
>
grounded_sage01:12:22

Hey there I am having a little trouble with changing the name of a file. I have tried doing an io/copy and a str/replace but neither seem to work. I'm not sure what the best way to go about it is. I basically want to take a template.html file and then change it to the path of a bidi route (eventually I will map the task over it..?). Anyways here is the current code.

grounded_sage01:12:27

(def next-route (path-for app/my-routes :community))

(defn render-template
  [in-file out-file]
  (doto out-file
    io/make-parents
    (spit (str/replace (slurp in-file) "{{CONTENT}}" "Happy times!"))))


(deftask generate-page []
  (let [tmp (tmp-dir!)]
    (with-pre-wrap [fileset]
      (empty-dir! tmp)
      (let [in-files (input-files fileset)
            template (by-name ["template.html"] in-files)]
        (doseq [in template]
          (let [in-file (tmp-file in)
                in-path (tmp-path in)
                out-path in-path
                out-file (io/file tmp out-path)]
            (render-template in-file out-file)))
        (-> fileset
            (add-resource tmp)
            commit!)))))

grounded_sage01:12:56

I did the io/copy at

(tmp-file (made-a-copy in one-being copied)
and I tried doing a string replace at
out-path (replace-name in-path "/template.html" next-route)

richiardiandrea02:12:10

@grounded_sage if I remember correctly there should be a move in the sift task

grounded_sage10:12:56

Unfortunately this hasn't seemed to have gotten me much further. I have attempted this

(defn change-name [path]
  (sift path :move{#"template.html" next-route}))
But still don't seem to be getting anywhere. I'm completely lost as I there seems to be very little beginner friendly documentation around Regex's and classpath files.

martinklepsch10:12:57

@grounded_sage sift is a task so you'll need to use it in the boot pipeline, like this (sift :move{#"template.html" next-route}) (note there's no path anymore)

grounded_sage10:12:45

@martinklepsch so is sift something I use inside of the generate-page task as I mentioned above? I currently have out-path (change-name in-path inside the let.

martinklepsch10:12:35

No you'd use them in the task pipeline like (comp (generate-page) (sift ...))

grounded_sage11:12:26

ok.. So essentially if I want to pick up the template.html from resources and map the bidi routes over it with the corresponding rum rendering. I would do something like map the bidi route handlers over (comp (generate-page (sift ...)))

grounded_sage11:12:50

Sorry if that sounds confusing as I actually don't even know what I am saying xD

martinklepsch11:12:28

generate-page returns a fileset with a file template.html in it. sift is a task that can modify the fileset. if you want template.html to be copied to multiple places you can do something like (comp (generate-page) (sift :copy {"template.html" "somepath.html"})) — except there is no :copy option for the sift task

grounded_sage11:12:50

He starts to wonder if it was worth building his own static site generator 😆

grounded_sage11:12:18

It seems like I would need to retain the original fileset with template.html. Then map over the bidi routes with the original fileset (generate-page) and (sift...) commiting each change. Then deleting template.html from the fileset when it is all done so it doesn't end up in the target folder.

armed15:12:24

Hey, folks. Is there any boot alternative for lein's :omit-sources option?

micha15:12:35

@armed is that to omit source files from a jar?

armed15:12:55

@micha exactly

micha15:12:36

ah yes, there is no option for that, but only files in :resource-paths will end up in jars

micha15:12:55

so if you add the source files you want to have on the classpath at build time to :source-paths they won't be in the jar

micha15:12:11

both :source-paths and :resource-paths are on the classpath

micha15:12:25

but only :resource-paths will be packeged by packaging type tasks

armed15:12:47

well, that’s strange. I’ve boot build-ed sample project from https://github.com/adzerk-oss/boot-uberjar-example and got all sources in project.jar.

micha15:12:56

also you can use the sift task to remove things from the fileset before the jar task

micha15:12:27

sure, look at line 4 of the build.boot

micha15:12:51

everything in the "src" dir will end up in the jar

armed15:12:57

Oh, thanks. I didn't look at first lines. Going to try now

micha15:12:00

but really in hindsight i think the sift task would be the way to go

micha15:12:21

rather than having :source-paths and :resource-paths

armed15:12:32

how to do it with sift, I couldn’t find option for deleting files.

micha15:12:51

you can use the :include option

armed15:12:03

include with revert?

micha15:12:04

and if you want to exclude you can use the :include option with :invert

armed15:12:05

@micha thanks for help. I need to familiarise with this a litle bit weird logic

micha15:12:04

which part is weird? the :invert thing?

micha15:12:57

it's like grep

armed15:12:08

ah, another question: what does exclude do in uber task? I tried to exclude sources with it.

armed15:12:31

but it didn’t help obviously

armed15:12:55

(uber :exclude #{#”.*clj"})

micha15:12:55

it excludes things from what it explodes from jar files

micha15:12:07

the uber task only adds things to the fileset

micha15:12:11

it can't remove things

micha15:12:33

the purpose of the uber task is simply to explode files from dependency jars and add them to the fileset

armed15:12:08

Ok, I got it. Thank you again!

micha15:12:55

it' skind of like the unix model

micha15:12:11

a collection of tasks that each do one thing

micha15:12:26

and you connect them in a pipeline

armed15:12:10

I see, it’s like grunt vs gulp

armed15:12:51

leiningen is grun, boot is gulp. I mostly do frontend stuff 🙂

micha15:12:14

are you familiar with the ember one, i think it's called broccoli?

micha15:12:27

boot seems to be very similar to that in some ways

micha15:12:36

especially with respect to how the tasks know which work needs to be done

armed15:12:54

I like unix way, but didn’t think about boot in that way.

armed15:12:21

Now all seems more clear

micha15:12:39

boot tasks are like unix processes, and the "stuff" that is piped from one to the next is the fileset

armed15:12:33

ok, right. fileset is like requestmap in ring

micha15:12:45

immutable

micha15:12:50

which is key

micha15:12:06

so a task can keep a reference to the fieset it gets

micha15:12:22

and later do things like compare it against another fileset

micha15:12:26

things like that

armed15:12:58

So I can write my own task, which modifies fileset by removing unwanted files from it, and passes to next task.

micha15:12:09

exactly yes

micha15:12:26

that's what the sift task is about

micha15:12:36

it has all kinds of fileset transformations

micha15:12:45

and that's all it does

armed15:12:53

whoa! It’s much-much better than lein.

armed16:12:29

@micha where I can look for structure of fileset map? What fields it has, etc.

micha16:12:02

the fileset is a record type

micha16:12:21

you would probably want to interact with it via protocol methods

micha16:12:34

yeah i recommend looking at https://github.com/boot-clj/boot/blob/master/doc/boot.core.md for api functions that do what you want

micha16:12:58

the structure of that record is not part of the official versioned api

mf16:12:41

@micha good to know

armed16:12:15

yep, you can change structure, but leave protocols untouched.

micha16:12:35

we won't change anything unless we really need to of course

mf16:12:01

How do I determine the roles that get applied to generated files in a tmp dir (tmp-dir!) when they are are added to the FS via add-resource?

micha16:12:27

"resource" things get both input and output roles

mf16:12:16

So given that they will have an output role as well, that will mean they will be emitted to the target dir?

mf16:12:32

As a general rule should all tasks that produce files emit to the target dir?

micha16:12:48

i think so yes

micha16:12:07

if a task creates a file, it's safe to assume that the user wants to include it in the artifact

micha16:12:25

this isn't always true, but i think is a reasonable default

mf16:12:59

I was envisaging a "pipeline" of tasks where each sub-task applied a transformation as part of the bigger whole

mf16:12:25

in general I don't see the user being interested in the intermittent states of the files

mf16:12:01

which I guess begs the question "why run as seperate tasks?"

micha16:12:29

separation of concerns is helpful sometimes

mf16:12:39

yeah that's what I figured

micha16:12:41

not always

micha16:12:13

i like having things decoupled because i don't like spending a lot of time on the build tooling

micha16:12:25

so if i can hack something together to get a project moving i'll do it

mf16:12:29

So I was unsure if each task should be writing to the target dir

micha16:12:33

separation of concerns really helps there

mf16:12:01

but I think the answer is that yes, if you add files to FS via add-resource it will be going in the output

micha16:12:07

tasks really shouldn't know about any named place in the filesystem at all

micha16:12:30

when you have a named place in the filesystem you introduce unmanaged shared global state

mf16:12:38

yeah I follow

mf16:12:43

I like this about Boot

micha16:12:46

oh i see what you mean

micha16:12:57

you weren't talking about tasks writing directly to the target directory

mf16:12:23

implicitly via the role that is applied

micha16:12:27

right yeah

micha16:12:41

the user can always use sift to remove unwanted things

micha16:12:12

and most of the time when a task creates a file that task can't know if it is an intermediate ssource or a final artifact

micha16:12:20

like some other task may process it further

micha16:12:26

or maybe not

micha16:12:09

so just having the default be to add thigns as resources seems at least consistent and easy to fix with sift later

mf16:12:22

So is it common for a task to remove files from the FS that have a output role?

mf16:12:58

i.e. if "task A" produces some files, and then "task B" produces some derived files

mf16:12:21

can "task B" remove the original files, produced by "task A"

micha16:12:30

it depends

micha16:12:47

if for example the original file is incompatible with the derived one

micha16:12:05

like sometimes you can't have two versions of something on the classpath maybe

micha16:12:09

things like that

mf16:12:28

It would be more like that the user would not be interested in the artifact of "task A", if they also ran "task B"

micha16:12:40

right yeah

mf16:12:15

Does it sound like I'm coming at this from a strange (non-Boot) perspective?

micha16:12:27

no, that all sounds very reasonable to

micha16:12:45

i think the user generally has some expectation for a given task

micha16:12:57

about how the task is expected to behave

micha16:12:04

hopefully it's intuitive

mf16:12:29

Is there a specific function for removing output files from the FS?

mf16:12:40

or is that what sift is for?

micha16:12:06

sift is a pre-made task you can use to do some commonly needed transformations

micha16:12:35

for things it doesn't support you can make your own task and use functions like boot.core/rm for exaple

micha16:12:41

to remove things from the fileset

mf16:12:04

ok, I'll take a look at rm

mf16:12:20

@micha really appreciate the help

micha16:12:58

like this (rm fileset (->> fileset output-files (by-re [#"\.clj$"])))

micha16:12:12

that would return a new fileset that doesn't contain any .clj output files

mf16:12:18

gottcha

micha16:12:34

then you'd probably want to commit! that before passing to the next task

mf16:12:40

yup will do

mf16:12:55

Thanks again @micha

micha16:12:59

sure any time

mattly22:12:41

has anyone successfully gotten boot_test's --junit-output-to working with CircleCI's test metadata collector?