Fork me on GitHub
#boot
<
2016-01-21
>
micha00:01:22

@seancorfield: that sounds like a pretty neat solution

jaredly00:01:48

oooh dunaj has type signatures!

richiardiandrea00:01:56

Hey guys, in boot, is there some util to remove the parent dir to a bunch of files with absolute path?

richiardiandrea00:01:10

Just asking for lazyness

micha00:01:20

like for example?

richiardiandrea00:01:15

I have /home/foo/bar.cljs and /home/bar/baz.clj...absolute, I would like to get foo/bar.cljs and bar/baz.clj

micha00:01:57

the best way, i think, is to use the java 7 nio functions

micha00:01:07

boot does have some functions but they don't add much

richiardiandrea00:01:49

do you know why? Because jgit returns of course relative paths (to the repo), but if I want to rename a file, I need an absolute path...so I transform everything...but then for add-resource, I need to normalize again everything back to relative

richiardiandrea01:01:03

ok I will have a look at it

micha01:01:11

(require '[ :as io])

(def root (.toPath (io/file "/home/foo")))
(def absolute (.toPath (io/file "/home/foo/bar.cljs")))

(-> root (.relativize absolute) .toString)     ;=> "bar.cljs"

micha01:01:51

using the java things makes it easy to write code that will work on windows and unix filesystems

micha01:01:02

you can make your own little clojure wrapper functions as you need them

richiardiandrea01:01:05

great I did not know relativize

micha01:01:21

there is also the opposite, .resolve()

micha01:01:13

(def root (.toPath (io/file "/home/foo")))
(def relative (.toPath (io/file "bar.cljs")))

(-> root (.resolve relative) .toString)    ;=> "/home/foo/bar.cljs"

micha01:01:56

another nice thing about the Path objects is that they carry info about the filesystem with them

micha01:01:08

so you can have a Path that is referring to an entry in a zip file

micha01:01:19

or a Path that is referring to an in-memory filesystem

micha01:01:21

and so on

micha01:01:25

and you can do like

micha01:01:38

(java.nio.file.Files/copy path1 path2)

micha01:01:48

and it doesn't matter which filesystems those paths are on

micha01:01:02

which is a pretty big advantage of the Path over the File object

richiardiandrea01:01:11

wow that's amazing, I had not explored the dark side of NIO at all 😄

jaredly01:01:12

looks like boot grabs everything in the “source” directly, regardless of whether it’s required by the “main” namespace… is there a way to make it just follow the requires, and leave out other things?

micha01:01:42

@jaredly: how do you mean?

micha01:01:00

nio is pretty incredible

micha01:01:18

i wish i had spent more time with it before starting with boot simple_smile

richiardiandrea01:01:44

is there a nio wrapper for Clojure already?

micha01:01:55

i think there are some, but it's not simple to wrap

micha01:01:02

there are a lot of permutations

micha01:01:11

so i think it's not really buying much

richiardiandrea01:01:15

well sometimes the best comes out of frustration, so it is good you did not know about it 😄

micha01:01:31

if you find one that's good let me know simple_smile

micha01:01:58

another nice thing about the nio api is that it's already a functional interface more or less

richiardiandrea01:01:18

really, Java? wow 😄

micha01:01:18

it's organized in two main java classes that just have static methods

micha01:01:29

java.nio.file.Paths and java.nio.file.Files

micha01:01:44

in there are just plain functions (static methods) that operate on Path objects

micha01:01:07

so like to copy one Path to another you just call a Java function:

micha01:01:21

(java.nio.file.Files/copy p1 p2)

richiardiandrea01:01:48

it would be great if Clojure 1.9 included some of this

richiardiandrea01:01:05

if they are already immutable and functional, why not?

micha01:01:11

you can use it in clojure, no problem

micha01:01:30

it's not available in java < 1.7 though

richiardiandrea01:01:53

yep but if Clojure would bump the Java dep, you could wrap and put as part of the Clojure API 😄

micha01:01:07

yep, one day

meow01:01:34

soon-ish

jaredly03:01:41

@micha maybe it’s a boot-cljs thing, or maybe it’s a clojurescript core thing, but I’ve got a .cljs file that I only want included in the bundle under a certain build, and not included when I run a different build. it’s causing problems b/c it has some top-level effects

jaredly05:01:48

looks like files in asset dirs don’t get watched + copied over when just using the normal (watch) task. is there a way to do that? I don’t see a config option for it

micha07:01:59

@jaredly: that should work

micha07:01:06

if not then it's a bug

martinklepsch09:01:09

@jaredly: one solution is to put it into a separate source directory and only add this directory to source-paths when you need it

martinklepsch09:01:48

@jaredly: do you use a .cljs.edn file? IIRC a shim namespace requiring all namespaces in your project is generated and used as main namespace (`:main` compiler option)

ska10:01:45

Hi. Is there somewhere an up to date example how to correctly create an uberjar? I need to deliver a runnable JAR and until recently everything worked fine. Since I introduced the target task, I only get a JAR called "project.jar" and the target directory still contains all the exploded classes. My expectation was, that after the jar task only the jar itself would be left in the file-set and that the JAR would be named after my project and version number. Probably I am doing something wrong. 😕

ska10:01:09

OK, reading the source of the jar task I take it, that the JAR is only added to the file-set. That explains the crowded target dir. Still somehow, the naming is weird for me.

martinklepsch10:01:02

@ska: when exploding other jars they may contain pom.properties/xml files this leads to the problem that you no longer can safely determine the right jar name

martinklepsch10:01:40

@ska: there is a file option to the jar task to manually set it in those cases

martinklepsch10:01:26

as for removing all files except the jar you can use sift somewhat like this (sift :include #"jarname\.jar")

martinklepsch10:01:53

:include removes all files from the fileset not matching the supplied regex

martinklepsch10:01:32

(If you ever want to exclude stuff use :include + :invert true.)

weavejester13:01:54

Quick question: is it permitted in boot to add custom keys to the env?

martinklepsch13:01:24

@weavejester: permitted yes, usually not ideal though

weavejester13:01:54

@martinklepsch: Thanks. I think the env is the best place for this. I need a way of persisting environment data into a pod.

weavejester13:01:06

The only other way I can think of is to use a file

martinklepsch13:01:34

what kind of data is it?

weavejester13:01:45

It’s for the boot implementation of environ

martinklepsch13:01:17

ah neat, so you're trying to fix the problem that env vars are not available in pods

weavejester13:01:54

Leiningen has a similar problem, but I use a hidden “.lein-env” for that.

weavejester13:01:24

I was thinking with boot I could use a key like: environ.boot/env

martinklepsch13:01:29

Why not take the same approach with boot?

weavejester13:01:57

So that’s another option. I could use a file to pass the data into the pod.

martinklepsch13:01:26

I.e. the first task invocation would write a file .environ and in newly created pods you look things up from that file if it exists

martinklepsch13:01:59

(where "write a file" = "add file to fileset/classpath")

weavejester13:01:07

Yeah, that’s basically what the PR does that I’m looking at right now. I was just wondering if there was a better way.

weavejester13:01:45

And as I looked into the source code for pods, the fact they persist env was tempting.

weavejester13:01:07

Or at least, it’s recommended they do.

martinklepsch13:01:41

I think putting things in boot's global env would work but people usually avoid it. Can't provide any useful examples where it might cause problems though. 😕

weavejester13:01:05

Maybe I’ll stick with a file for now.

weavejester13:01:12

It seems the safer option.

martinklepsch13:01:57

ah one comes to mind: by putting it in env you rely on people enrichting (get-env) vs creating a new env themselves

martinklepsch13:01:37

most of the time pods are created that way but still it's an assumption and not the rule.

meow13:01:37

@weavejester is in the house! 😈

weavejester13:01:16

Seems like files are the way to go, then.

weavejester13:01:33

Incidentally, Boot is looking nicer and nicer.

martinklepsch13:01:06

@weavejester: I should do more PRs to your projects your feedback is always fantastic simple_smile

weavejester14:01:17

@martinklepsch: Well, there are some PRs that I manage to forget about, so I’m not sure about “always” simple_smile

martinklepsch14:01:12

@weavejester: fwiw not much has changed about boot since 2.0, most stuff is improvements and fixes simple_smile

martinklepsch14:01:33

@weavejester: ok then lets say: if you manage to give feedback, it's always great 😉

weavejester14:01:50

Haha, maybe I’ll accept that simple_smile

martinklepsch14:01:15

@weavejester: I'm thinking the task should write the file by default, there's not much reason not to do it + it eases usage when people add their tests etc (or other things that access env vars from within pods)

weavejester14:01:14

@martinklepsch: I’ll go with that option then

weavejester14:01:31

@martinklepsch: Thanks for taking a look over the PR yourself.

jgdavey14:01:22

@weavejester: I use environ (but not the lein plugin) with boot, and I haven’t had any issues. I’m not using ENV vars in pods, though. Dev-time-only stuff is configured in build.boot.

jgdavey14:01:05

It’s a bit odd to have a file called .lein-env in a boot project, but other than that, it’s seemless.

alandipert14:01:42

are jvm system properties not an option? they have the merit of jvm-globalness

jgdavey14:01:05

environ looks at all 3

weavejester14:01:20

@jgdavey: The .lein-env file should only appear if you’re using Leiningen.

weavejester14:01:40

Unless you’re adding it in manually?

jgdavey14:01:41

Oh, I typically create one with a map of stuff I want.

weavejester14:01:19

Environ has a boot task, but currently it doesn’t work in pods.

weavejester14:01:38

The .lein-env file is really just designed to transfer stuff from lein into the new environment.

jgdavey14:01:19

It follows well the “12-factor” style of application. I use .lein-env for dev, and environment variable (and sometimes system properties) for prod

jaredly14:01:29

@martinklepsch: is the .cljs.edn stuff documented anywhere? I’m currently using the :main option directly

martinklepsch14:01:27

that it's still under the multiple builds section is a bit misleading

martinklepsch14:01:38

boot-cljs could use a documentation makeover 😛

jaredly15:01:17

how does boot find the .cljs.edn file? Do they have to be in asset-paths, or resource-paths?

martinklepsch15:01:00

@jaredly: resource or source paths is best, might even work with asset paths

jaredly15:01:28

and it just looks for all .cljs.edn files, and makes a build for each?

martinklepsch15:01:36

@jaredly: exactly

jaredly15:01:08

interesting

ska15:01:21

@martinklepsch: Sorry to be late. You answered my question earlier saying that several POMs (after exploding JARs) would confuse boot making it impossible to compute the correct filename. 1) It used to work earlier? Why was this changed? 2) Does this mean, that the steps of a task can only ever communicate via files and env and that the information of the project itself is not passed along from step to step?

martinklepsch15:01:00

@ska: usually when making a jar the pom task and the jar task talk to each other via the pom.xml file. The jar tasks extracts the name and version and uses it for the filename. Boot does not really have the concept of a "project".

martinklepsch15:01:44

@ska: I think earlier the code was "if there are multiple pom.xml, use the first" now it's "if there are multiple, don't assume things"

pesterhazy15:01:00

is there a template for just spawning a process with boot?

pesterhazy15:01:09

like a helper function simple_smile

martinklepsch15:01:36

@pesterhazy: you mean like (sh "ls -la")?

pesterhazy16:01:52

no idea how I missed that

ska16:01:03

@martinklepsch: OK, I understand the change to that is does not work anymore. Makes sense. The fact that the project information is not available throughout the whole pipeline seems weird to me. That information is quite central in the scope of a project. Is there some deeper thinking behind this that I don't see yet or was there just no time to do it?

micha16:01:44

boot has no internal concept of a "project", because it's not a build tool simple_smile

martinklepsch16:01:00

@ska: I think the point is just that "project" is an implementation detail of making jars and boot isn't bothering with it simple_smile

micha16:01:06

if you want to you can just def a var to contain that information

martinklepsch16:01:15

Like where else do you need project information than for making the jar?

micha16:01:34

also the pom is the canonical project descriptor for java/clojure

micha16:01:46

but you may be using boot to build something else

micha16:01:04

so using the pom seems like the most correct behavior

micha16:01:34

like if you use boot to build an npm module or something, the package.json file would be the canonical project descriptor

micha16:01:46

and it might have different semantics than the pom

micha16:01:09

so tying boot to a specific thing like the pom structure just seemed unnecessary and problematic

micha16:01:56

so i recommend inspecting the pom.xml wherever you would use the project info, because the pom must contain the info you need

ska16:01:13

Now, you've lost me. It is tying to the POM by using that to pass information from step to step instead of providing an abstraction on it.

micha16:01:44

the pom is the abstraction

micha16:01:53

there is nothing to add

micha16:01:25

there are plenty of functions included in boot to extract info from the pom: https://github.com/boot-clj/boot/blob/master/doc/boot.pod.md#jars

ska16:01:26

Hmmm... "Boot is a Clojure build framework [...]. Boot provides a runtime environment that includes all of the tools needed to build Clojure projects from scripts written in Clojure that run in the context of the project." --> to me that sounds like package.json is not a use case and it implies the project context.

micha16:01:48

that wasn't the intention, anyway

ska16:01:52

Don't get me wrong. I don't want to be nitpicking. simple_smile

micha16:01:04

we didn't want to start hardcoding a particular build structure into boot

micha16:01:49

the pom is the fundamental abstraction here, and there is nothing you can add to it because it's constrained by the maven requirements

micha16:01:56

so why not just use it?

ska16:01:20

Maybe all my question would go away, if there was a built-in uberjar tasks. It seems too hard to do such a simple thing.

micha16:01:20

there is also nothing preventing you from doing like

micha16:01:34

what's hard about it?

micha16:01:56

consider:

micha16:01:10

(def project-info {:project 'foo/bar :version "1.2.3"})

(let [{:keys [project version]} project-info]
  (task-options!
    jar {:file (format "%s-%s.jar" (name project) version)}
    pom {:project project :version version}))

micha16:01:27

you can of course make a library that does that for you, if you want

micha16:01:36

we did a similar thing with bootlaces

micha16:01:25

but one of the things i like about boot is that it totally works without a project context

micha16:01:42

so you can have self-contained scripts that do things

micha16:01:44

and so on

ska16:01:13

Maybe I am still in Maven land and not yet ready for gradle. Oh, wait, that's leiningen and boot. 😉

dave16:01:22

PSA: i just pushed v1.1.0 of https://github.com/adzerk-oss/boot-jar2bin -- there is now a --jvm-opt flag so you can set JVM options for your executables e.g.

boot aot pom uber jar bin --jvm-opt -Dclojure.compiler.direct-linking=true

micha16:01:39

do you see how the example i pasted above could solve your problem?

micha16:01:59

note that task-options! is setting defaults

micha16:01:15

you can call task-options! again and settings you give there will override those

ska16:01:35

@micha: Yes, and I already considered going down that route. I wasn't sure about dynamically setting things, yet. And it somehow felt like re-implementing widespread defaults.

micha16:01:55

you can make your own defaults as wide-spread in your world as you like simple_smile

micha16:01:06

and publish your library for others to use

ska16:01:19

But I will just do it like that for the time being to be able to go on with the project itself.

ska16:01:35

Thanks for taking the time to discuss this.

micha16:01:03

i guess our thinking was that if we decided on a project structure, then you wouldn't be able to do this kind of thing

ska16:01:04

Over there it is just a "lein uberjar"... ho, humm.

micha16:01:37

yes but then you need to work completely within the constraints of that implementation

micha16:01:08

for example if you want to make an uberjar that has exclusions that can't be expressed as regex, can you do it?

micha16:01:27

with boot you can just remove things from the fileset arbitrarily between the uber and jar tasks

micha16:01:32

or add things

micha16:01:08

(deftask uberjar
  []
  (comp (uber) (jar)))

ska16:01:17

Sure, I am aware of the benefits. No need to convince me.

ska16:01:42

But that task will create the wrong filename.

ska16:01:59

(unless set before, of course)

micha16:01:05

i mean project context doesn't really make sense with uberjars, imo

micha16:01:16

you should not be putting uberjars in maven

ska16:01:28

I am delivering an uberjar to a customer.

micha16:01:30

so project and version are somewhat meaningless, or at least arbitrary

micha16:01:48

it's not a dependency coordinate, is what i mean

micha16:01:53

it's just a filename

ska16:01:13

Sure, but it's good knowing which version you received and are starting anyway.

micha16:01:10

i'd be interested in hearing ideas about how to differentiate the project pom from poms pulled in by uber task

ska16:01:27

Anyway. I have a solution. Maybe I will come to the conclusion later that it is worth the extra effort. Maybe I won't. For this project I might switch back to leiningen, who knows.

ska16:01:52

I certainly agree with a lot of ideas in boot.

pesterhazy16:01:37

Question about task order with comp: In https://github.com/pesterhazy/boot-react-native/blob/feature/cli-ios-sim/example/build.boot I'd like to make a task run after rn/after-cljsbuild

pesterhazy16:01:24

but if I add a task (`with-pre-wrap`) at the end of the comp, it gets run at the wrong time

martinklepsch16:01:32

put it before target. but also be aware that the middleware format allows you to run code/initialize tasks before they're actually run

pesterhazy16:01:10

(deftask run-in-simulator
  "Run the app in the simulator"
  []
  (let [running (atom false)]
    (c/with-pre-wrap fileset
      (println "**** run-in-simulator:" @running)
      (when-not @running
        (reset! running true)
        #_(binding [util/*sh-dir* "app"]
            (util/dosh "node" "node_modules/react-native/local-cli/cli.js" "run-ios")))
      fileset)))

pesterhazy16:01:34

that's my code. But the println gets executed at the wrong time

micha16:01:49

@ska perhaps the solution is for the uber task to put metadata on the files it extracts into the fileset

micha16:01:59

so other tasks can know that they're "foreign" files

micha16:01:31

then the multiple poms issue would have an easy solution, the jar task could ignore those poms from dependency jars

pesterhazy16:01:53

@martinklepsch: I think the reason may be that after-cljsbuild delays its work?

pesterhazy16:01:04

it uses with-post-wrap

ska16:01:40

Hah! I just wrote my own defn jarname ... guess what? It is already there. In boot.util.

martinklepsch16:01:46

@ska: did you use :refer :all? And did you just say it loud and clear? 😛

martinklepsch16:01:31

@pesterhazy: There are multiple with-pre/post-wrap tasks in the react-native thing — should it be ran after the packager is being ran?

martinklepsch16:01:39

maybe try with-post-wrap

pesterhazy16:01:42

@martinklepsch: yes

pesterhazy16:01:45

that seems to work

martinklepsch16:01:01

but it's more of a guess, the react-native tasks are a bit hard to follow

pesterhazy16:01:02

but then I need to move run-in-simulator to the top of the chain

pesterhazy16:01:08

yup that works simple_smile

ska16:01:36

@martinklepsch: No, I did no such thing. It was there without me doing anything. At least not on purpose.

micha16:01:36

the boot.util namespace is referred into the build.boot namespace (i.e. boot.user)

martinklepsch16:01:56

oh, didn't know haha

martinklepsch16:01:00

sorry then @ska simple_smile

ska16:01:47

@martinklepsch: If you witness me doing a :refer :all anytime (other that core.async) I owe you a beer. simple_smile

martinklepsch16:01:01

I'll keep that in mind 😄

martinklepsch16:01:25

goes to github searches really old github projects of U066F9CUW using :refer :all

ska17:01:16

The really old stuff probably still uses :use but that does not count!

micha17:01:47

later @ska!

prasincs19:01:46

Hey, what’s a good way to not include certain files, directories from your path in resources for making uberjar?

micha19:01:19

hi @prasincs !

micha19:01:41

there are --exclude and --include options

micha19:01:05

you can also use the sift task between uber and jar to do more complex manipulations

prasincs19:01:54

yeah, I tried it a while back.. is there an example? I want to only include say lib/resources/.+/dir in uberjar and ignore the other directories in those directories

prasincs19:01:56

— additionally, has someone done any work around deduping depdendencies or removing classes not being used from jars?

micha19:01:40

boot uber --include 'lib/resources/.+/dir' show -f

micha19:01:05

the show -f lets you see how the fileset looks after the uber

micha19:01:21

it'll print a tree to the console

prasincs19:01:36

can I just run show without doing a build?

prasincs19:01:40

like a dry run

micha19:01:20

it just prints a tree of whatever fileset it gets

prasincs19:01:45

how does that translate into code?

prasincs19:01:51

task-opts! ?

alandipert19:01:52

boot uber --include 'lib/resources/.+/dir' show -f == (boot (uber :include #{#"lib/resources/.+/dir"}) (show :fileset true))

prasincs19:01:35

ah, thanks @alandipert

alandipert19:01:07

np. w/ code is a faster way to develop task stacks in the repl... you can do boot repl and drive the build that way

alandipert19:01:29

if/when you update your build.boot, you can reload it in the repl with (load-file "build.boot")

prasincs19:01:23

how do I reset the repl setup in this case? or does it do that itself? [I’d like to think boot does the right thing and doesn’t set any globals, just checking :)]

prasincs19:01:37

between builds or creating stacks

alandipert19:01:08

well, developing on the build.boot itself is kind of a special case

alandipert19:01:29

any tasks you define in build.boot are global

alandipert19:01:13

but unless you define new ones, no, nothing is retained between boot calls

micha19:01:25

functions with a bang on the end, like task-options! do mutate state

micha19:01:57

but most of them, like task-options! in this case, are idempotent

prasincs21:01:34

thanks @micha

richiardiandrea23:01:02

hello guys, is there a magic way to get the jar files given a dependency inside boot (maybe part of aether.clj?)

micha23:01:31

that's if you need the jar file for a single dependency

richiardiandrea23:01:22

awesome, I don't even need to map 😄

richiardiandrea23:01:42

basically I need to fetch the source jar, if present

micha23:01:50

yep that will work

richiardiandrea23:01:04

so yep, I just need to add _source