Fork me on GitHub
#boot
<
2016-09-21
>
kenbier00:09:25

is copy-files a kosher way to copy one directory to another in a task? https://github.com/boot-clj/boot/blob/2.5.2/boot/pod/src/boot/file.clj#L172

kenbier00:09:11

something like;

kenbier00:09:26

(deftask foo []
  (fn [next-task]
    (fn [file-set]
      (boot-file/copy-files "public/" (str "public/" (env :ver 0) "/"))
      (next-task file-set))))

micha02:09:27

sure, the "public" directory isn't in the fileset right?

kenbier02:09:22

@micha “public/“ is in resources/public. and my :resource-path is set to “resources”.

kenbier02:09:32

so i suppose it is in the fileset ?

micha02:09:35

ah so you don't want to do that

micha02:09:42

you can use the sift task for that

kenbier02:09:50

but i want to copy it, not move it

kenbier02:09:19

i plan to add a tag to that directory in each release, and sync it to s3

kenbier02:09:25

but i also want to leave the public/ intact

kenbier02:09:34

and from what i understand :sift does not have a copy

kenbier02:09:52

i did find a boot-copy task but it seems its not maintained and is using deprecated api 😕

kenbier02:09:17

got a better suggestion on how to copy everything from public -> public/some-number/ ?

micha02:09:56

ah, the sift task should have a copy

micha02:09:00

that's a bummer

micha02:09:35

there is the boot.core/cp function

kenbier02:09:18

hm is the bug only in 2.6? we are on 2.5.2 atm

micha02:09:13

ah 2.5.5 is ok, so 2.5.2 is also presumably

micha02:09:38

we really want a copy option to the sift task

micha02:09:55

the easiest thing to do in your case i think is to make a temp dir, copy the files into there

micha02:09:00

and add that to the fileset

micha02:09:17

you will then have duplicate copies of the file in different paths

micha02:09:28

if you want to remove the original you can do that by removing them from the fileset

micha02:09:09

you can use boot.core/rm to remove them

micha02:09:17

so like this, in sketch

micha02:09:52

(deftask foo []
  (let [tmp (tmp-dir!)]
    (with-pre-wrap fs
      (let [ins (->> fs
                     (output-files)
                     (by-re #{#"^public/"}))]
        (empty-dir! tmp)
        (doseq [in ins]
          (io/copy (tmp-file in) (io/file tmp "public" tag)))
        (-> fs (add-resource tmp) (rm ins) commit!)))))

micha02:09:59

you can improve on this by caching the previous fileset object and diffing it

micha02:09:09

then you don't need to copy all the files each iteration

micha02:09:13

only the ones that chanegd

micha02:09:40

but the main idea is you make a temp directory where you can copy your files to

micha02:09:48

with different paths

micha02:09:02

then you add that to the fileset and remove the original files from the fileset

micha02:09:24

and finally you commit! to sync with the underlying filesystem and pass the result to the next task

micha02:09:55

oh sorry, i'm dumb, you said you didn't want to move them

micha02:09:04

so disregard the stuff about removing the originals

kenbier02:09:11

yeah the same but no rm right?

kenbier02:09:31

sweet, thanks alot @micha !! helpful as always 🍪

micha02:09:37

one sec there is a bug

kenbier02:09:53

oh another question, what if i only need these copied file to be present for another boot task called bar? instead of committing them, could i run barafter altering the filesystem?

kenbier02:09:27

so it runs in the context of this modified filesystem, but its not one we pass to the next task? or is that an anitpattern?

kenbier02:09:52

i can live without it, just curious

micha02:09:07

(deftask foo []
  (let [tmp (tmp-dir!)]
    (with-pre-wrap fs
      (let [ins (->> fs
                     (input-files)
                     (by-re #{#"^public/"}))]
        (empty-dir! tmp)
        (doseq [in ins]
          (let [[path file] ((juxt tmp-path tmp-file) in)
                relpath (.replaceAll path "^public/" "")]
            (io/copy file (io/file tmp "public" tag relpath))))
        (-> fs (add-resource tmp) commit!)))))

micha02:09:48

in general it's good to design things so that tasks are as decoupled as possible

micha02:09:53

makes thinsg more hackable

micha02:09:13

the fileset pattern is what makes that possible in most cases

kenbier02:09:48

i see, that makes sense. its like ring where we wrap the middleware?

micha02:09:03

right, so you can separate concerns in the different layers

kenbier02:09:19

this should be fine

kenbier02:09:40

what does the relpath part do in the snippet above?

micha02:09:42

the bug in the first one was not handling the relative path underneath public

micha02:09:06

because you need to transform "public/foo.clj" --> "public/1.0/foo.clj" right?

micha02:09:31

yeah the relpath part is handling the "foo.clj" part of the path

micha02:09:39

but there is one more bug 🙂

micha02:09:08

you need to use io/make-parents to ensure that the subdirectories exist

micha02:09:20

before trying to write to the copy output file

micha02:09:50

so like (io/copy file (doto (io/file tmp "public" tag relpath) io/make-parents)))

micha02:09:03

i always forget that

kenbier02:09:43

ah yes, makes sense

micha02:09:48

if you'd like to make a github issue for the sift --copy option, we can make it happen for 2.7.0

kenbier02:09:55

sure, ill do that

micha02:09:08

thanks! 💥

alandipert20:09:42

@richiardiandrea made https://github.com/micha/json-table/tree/key-wildcard if you're interested - adds . and ^k operators to jt

richiardiandrea20:09:11

@alandipert: I am ! I will check it out

domkm23:09:03

Is there a way to get the latest version of a dependency (including snapshots)? "LATEST" seems to ignore snapshots.

micha23:09:13

there is a trick: "(0,)" @domkm

richiardiandrea23:09:27

@alandipert you still need to have a % at the end for "gets" right?

richiardiandrea23:09:53

kk I should patch that 😄

micha23:09:00

there was a problem with it

micha23:09:06

i wrote it up in the branch

richiardiandrea23:09:12

ooooh there it is ... yeah makes sense...

richiardiandrea23:09:09

so up to there user to discern what he/she wants and less logic in the code, it makes sense, less ambiguity

micha23:09:22

yeah we use it for scripts that need to be robust to weird json

micha23:09:47

so it's good to know exactly what will be output

richiardiandrea23:09:23

the key wildcard looks great for exploring