@seancorfield: that sounds like a pretty neat solution

oooh dunaj has type signatures!

Hey guys, in boot, is there some util to remove the parent dir to a bunch of files with absolute path?

Just asking for lazyness

like for example?

I have /home/foo/bar.cljs and /home/bar/baz.clj...absolute, I would like to get foo/bar.cljs and bar/baz.clj

oh sure

the best way, i think, is to use the java 7 nio functions

boot does have some functions but they don't add much

do you know why? Because jgit returns of course relative paths (to the repo), but if I want to rename a file, I need an absolute path...so I transform everything...but then for add-resource, I need to normalize again everything back to relative

ok I will have a look at it

(require '[ :as io])

(def root (.toPath (io/file "/home/foo")))
(def absolute (.toPath (io/file "/home/foo/bar.cljs")))

(-> root (.relativize absolute) .toString)     ;=> "bar.cljs"

oh

using the java things makes it easy to write code that will work on windows and unix filesystems

you can make your own little clojure wrapper functions as you need them

great I did not know relativize

there is also the opposite, .resolve()

good !

thanks!

(def root (.toPath (io/file "/home/foo")))
(def relative (.toPath (io/file "bar.cljs")))

(-> root (.resolve relative) .toString)    ;=> "/home/foo/bar.cljs"

another nice thing about the Path objects is that they carry info about the filesystem with them

so you can have a Path that is referring to an entry in a zip file

or a Path that is referring to an in-memory filesystem

and so on

and you can do like

(java.nio.file.Files/copy path1 path2)

and it doesn't matter which filesystems those paths are on

which is a pretty big advantage of the Path over the File object

wow that's amazing, I had not explored the dark side of NIO at all 😄

looks like boot grabs everything in the “source” directly, regardless of whether it’s required by the “main” namespace… is there a way to make it just follow the requires, and leave out other things?

@jaredly: how do you mean?

nio is pretty incredible

i wish i had spent more time with it before starting with boot

is there a nio wrapper for Clojure already?

i think there are some, but it's not simple to wrap

there are a lot of permutations

so i think it's not really buying much

well sometimes the best comes out of frustration, so it is good you did not know about it 😄

ok

if you find one that's good let me know

another nice thing about the nio api is that it's already a functional interface more or less

really, Java? wow 😄

it's organized in two main java classes that just have static methods

java.nio.file.Paths and java.nio.file.Files

in there are just plain functions (static methods) that operate on Path objects

so like to copy one Path to another you just call a Java function:

(java.nio.file.Files/copy p1 p2)

it would be great if Clojure 1.9 included some of this

if they are already immutable and functional, why not?

you can use it in clojure, no problem

it's not available in java < 1.7 though

yep but if Clojure would bump the Java dep, you could wrap and put as part of the Clojure API 😄

yep, one day

soon-ish

@micha maybe it’s a boot-cljs thing, or maybe it’s a clojurescript core thing, but I’ve got a .cljs file that I only want included in the bundle under a certain build, and not included when I run a different build. it’s causing problems b/c it has some top-level effects

looks like files in asset dirs don’t get watched + copied over when just using the normal (watch) task. is there a way to do that? I don’t see a config option for it

@jaredly: that should work

if not then it's a bug

@jaredly: one solution is to put it into a separate source directory and only add this directory to source-paths when you need it

@jaredly: do you use a .cljs.edn file? IIRC a shim namespace requiring all namespaces in your project is generated and used as main namespace (`:main` compiler option)

Hi. Is there somewhere an up to date example how to correctly create an uberjar? I need to deliver a runnable JAR and until recently everything worked fine. Since I introduced the target task, I only get a JAR called "project.jar" and the target directory still contains all the exploded classes. My expectation was, that after the jar task only the jar itself would be left in the file-set and that the JAR would be named after my project and version number. Probably I am doing something wrong. 😕

OK, reading the source of the jar task I take it, that the JAR is only added to the file-set. That explains the crowded target dir. Still somehow, the naming is weird for me.

@ska: when exploding other jars they may contain pom.properties/xml files this leads to the problem that you no longer can safely determine the right jar name

@ska: there is a file option to the jar task to manually set it in those cases

as for removing all files except the jar you can use sift somewhat like this (sift :include #"jarname\.jar")

:include removes all files from the fileset not matching the supplied regex

(If you ever want to exclude stuff use :include + :invert true.)

Quick question: is it permitted in boot to add custom keys to the env?

@weavejester: permitted yes, usually not ideal though

@martinklepsch: Thanks. I think the env is the best place for this. I need a way of persisting environment data into a pod.

The only other way I can think of is to use a file

what kind of data is it?

It’s for the boot implementation of environ

ah neat, so you're trying to fix the problem that env vars are not available in pods

Exactly

Leiningen has a similar problem, but I use a hidden “.lein-env” for that.

I was thinking with boot I could use a key like: ::environ.boot/env

Why not take the same approach with boot?

So that’s another option. I could use a file to pass the data into the pod.

I.e. the first task invocation would write a file .environ and in newly created pods you look things up from that file if it exists

(where "write a file" = "add file to fileset/classpath")

Yeah, that’s basically what the PR does that I’m looking at right now. I was just wondering if there was a better way.

And as I looked into the source code for pods, the fact they persist env was tempting.

Or at least, it’s recommended they do.

I think putting things in boot's global env would work but people usually avoid it. Can't provide any useful examples where it might cause problems though. 😕

Maybe I’ll stick with a file for now.

It seems the safer option.

ah one comes to mind: by putting it in env you rely on people enrichting (get-env) vs creating a new env themselves

most of the time pods are created that way but still it's an assumption and not the rule.

@weavejester is in the house! 😈

Seems like files are the way to go, then.

Incidentally, Boot is looking nicer and nicer.

+ :thumbsup:

@weavejester: I should do more PRs to your projects your feedback is always fantastic

@martinklepsch: Well, there are some PRs that I manage to forget about, so I’m not sure about “always”

@weavejester: fwiw not much has changed about boot since 2.0, most stuff is improvements and fixes

@weavejester: ok then lets say: if you manage to give feedback, it's always great 😉

Haha, maybe I’ll accept that

@weavejester: I'm thinking the task should write the file by default, there's not much reason not to do it + it eases usage when people add their tests etc (or other things that access env vars from within pods)

@martinklepsch: I’ll go with that option then

@martinklepsch: Thanks for taking a look over the PR yourself.

bbl

cheers

@weavejester: I use environ (but not the lein plugin) with boot, and I haven’t had any issues. I’m not using ENV vars in pods, though. Dev-time-only stuff is configured in build.boot.

It’s a bit odd to have a file called .lein-env in a boot project, but other than that, it’s seemless.

are jvm system properties not an option? they have the merit of jvm-globalness

environ looks at all 3

@jgdavey: The .lein-env file should only appear if you’re using Leiningen.

Unless you’re adding it in manually?

Oh, I typically create one with a map of stuff I want.

Ah, okay.

Environ has a boot task, but currently it doesn’t work in pods.

Gotcha.

The .lein-env file is really just designed to transfer stuff from lein into the new environment.

It follows well the “12-factor” style of application. I use .lein-env for dev, and environment variable (and sometimes system properties) for prod

@martinklepsch: is the .cljs.edn stuff documented anywhere? I’m currently using the :main option directly

https://github.com/adzerk-oss/boot-cljs/wiki/Usage#multiple-builds

that it's still under the multiple builds section is a bit misleading

boot-cljs could use a documentation makeover 😛

how does boot find the .cljs.edn file? Do they have to be in asset-paths, or resource-paths?

@jaredly: resource or source paths is best, might even work with asset paths

and it just looks for all .cljs.edn files, and makes a build for each?

@jaredly: exactly

interesting

@martinklepsch: Sorry to be late. You answered my question earlier saying that several POMs (after exploding JARs) would confuse boot making it impossible to compute the correct filename. 1) It used to work earlier? Why was this changed? 2) Does this mean, that the steps of a task can only ever communicate via files and env and that the information of the project itself is not passed along from step to step?

@ska: usually when making a jar the pom task and the jar task talk to each other via the pom.xml file. The jar tasks extracts the name and version and uses it for the filename. Boot does not really have the concept of a "project".

@ska: I think earlier the code was "if there are multiple pom.xml, use the first" now it's "if there are multiple, don't assume things"

reference: https://github.com/boot-clj/boot/issues/278#issuecomment-137436361

is there a template for just spawning a process with boot?

like a helper function

@pesterhazy: you mean like (sh "ls -la")?

https://github.com/boot-clj/boot/blob/master/doc/boot.util.md#shell

oh. yes

no idea how I missed that

@martinklepsch: OK, I understand the change to that is does not work anymore. Makes sense. The fact that the project information is not available throughout the whole pipeline seems weird to me. That information is quite central in the scope of a project. Is there some deeper thinking behind this that I don't see yet or was there just no time to do it?

boot has no internal concept of a "project", because it's not a build tool

@ska: I think the point is just that "project" is an implementation detail of making jars and boot isn't bothering with it

if you want to you can just def a var to contain that information

Like where else do you need project information than for making the jar?

also the pom is the canonical project descriptor for java/clojure

but you may be using boot to build something else

so using the pom seems like the most correct behavior

like if you use boot to build an npm module or something, the package.json file would be the canonical project descriptor

and it might have different semantics than the pom

so tying boot to a specific thing like the pom structure just seemed unnecessary and problematic

so i recommend inspecting the pom.xml wherever you would use the project info, because the pom must contain the info you need

Now, you've lost me. It is tying to the POM by using that to pass information from step to step instead of providing an abstraction on it.

right

the pom is the abstraction

there is nothing to add

there are plenty of functions included in boot to extract info from the pom: https://github.com/boot-clj/boot/blob/master/doc/boot.pod.md#jars

Hmmm... "Boot is a Clojure build framework [...]. Boot provides a runtime environment that includes all of the tools needed to build Clojure projects from scripts written in Clojure that run in the context of the project." --> to me that sounds like package.json is not a use case and it implies the project context.

that wasn't the intention, anyway

Don't get me wrong. I don't want to be nitpicking.

we didn't want to start hardcoding a particular build structure into boot

the pom is the fundamental abstraction here, and there is nothing you can add to it because it's constrained by the maven requirements

so why not just use it?

Maybe all my question would go away, if there was a built-in uberjar tasks. It seems too hard to do such a simple thing.

there is also nothing preventing you from doing like

what's hard about it?

consider:

(def project-info {:project 'foo/bar :version "1.2.3"})

(let [{:keys [project version]} project-info]
  (task-options!
    jar {:file (format "%s-%s.jar" (name project) version)}
    pom {:project project :version version}))

you can of course make a library that does that for you, if you want

we did a similar thing with bootlaces

but one of the things i like about boot is that it totally works without a project context

so you can have self-contained scripts that do things

and so on

Maybe I am still in Maven land and not yet ready for gradle. Oh, wait, that's leiningen and boot. 😉

PSA: i just pushed v1.1.0 of https://github.com/adzerk-oss/boot-jar2bin -- there is now a --jvm-opt flag so you can set JVM options for your executables e.g.

boot aot pom uber jar bin --jvm-opt -Dclojure.compiler.direct-linking=true

do you see how the example i pasted above could solve your problem?

note that task-options! is setting defaults

you can call task-options! again and settings you give there will override those

@micha: Yes, and I already considered going down that route. I wasn't sure about dynamically setting things, yet. And it somehow felt like re-implementing widespread defaults.

you can make your own defaults as wide-spread in your world as you like

and publish your library for others to use

But I will just do it like that for the time being to be able to go on with the project itself.

@ska, maybe adding a snippet might help? https://github.com/boot-clj/boot/wiki/Snippets

Thanks for taking the time to discuss this.

sure

i guess our thinking was that if we decided on a project structure, then you wouldn't be able to do this kind of thing

Over there it is just a "lein uberjar"... ho, humm.

yes but then you need to work completely within the constraints of that implementation

for example if you want to make an uberjar that has exclusions that can't be expressed as regex, can you do it?

with boot you can just remove things from the fileset arbitrarily between the uber and jar tasks

or add things

also:

(deftask uberjar
  []
  (comp (uber) (jar)))

lol

Sure, I am aware of the benefits. No need to convince me.

But that task will create the wrong filename.

(unless set before, of course)

i mean project context doesn't really make sense with uberjars, imo

you should not be putting uberjars in maven

I am delivering an uberjar to a customer.

so project and version are somewhat meaningless, or at least arbitrary

it's not a dependency coordinate, is what i mean

it's just a filename

Sure, but it's good knowing which version you received and are starting anyway.

i'd be interested in hearing ideas about how to differentiate the project pom from poms pulled in by uber task

Anyway. I have a solution. Maybe I will come to the conclusion later that it is worth the extra effort. Maybe I won't. For this project I might switch back to leiningen, who knows.

I certainly agree with a lot of ideas in boot.

Question about task order with comp: In https://github.com/pesterhazy/boot-react-native/blob/feature/cli-ios-sim/example/build.boot I'd like to make a task run after rn/after-cljsbuild

but if I add a task (`with-pre-wrap`) at the end of the comp, it gets run at the wrong time

put it before target. but also be aware that the middleware format allows you to run code/initialize tasks before they're actually run

(deftask run-in-simulator
  "Run the app in the simulator"
  []
  (let [running (atom false)]
    (c/with-pre-wrap fileset
      (println "**** run-in-simulator:" @running)
      (when-not @running
        (reset! running true)
        #_(binding [util/*sh-dir* "app"]
            (util/dosh "node" "node_modules/react-native/local-cli/cli.js" "run-ios")))
      fileset)))

that's my code. But the println gets executed at the wrong time

@ska perhaps the solution is for the uber task to put metadata on the files it extracts into the fileset

so other tasks can know that they're "foreign" files

then the multiple poms issue would have an easy solution, the jar task could ignore those poms from dependency jars

@martinklepsch: I think the reason may be that after-cljsbuild delays its work?

it uses with-post-wrap

Hah! I just wrote my own defn jarname ... guess what? It is already there. In boot.util.

@ska: did you use :refer :all? And did you just say it loud and clear? 😛

@pesterhazy: There are multiple with-pre/post-wrap tasks in the react-native thing — should it be ran after the packager is being ran?

maybe try with-post-wrap

@martinklepsch: yes

that seems to work

but it's more of a guess, the react-native tasks are a bit hard to follow

but then I need to move run-in-simulator to the top of the chain

yup that works

@martinklepsch: No, I did no such thing. It was there without me doing anything. At least not on purpose.

the boot.util namespace is referred into the build.boot namespace (i.e. boot.user)

oh, didn't know haha

sorry then @ska

@martinklepsch: If you witness me doing a :refer :all anytime (other that core.async) I owe you a beer.

I'll keep that in mind 😄

goes to github searches really old github projects of @ska using :refer :all

The really old stuff probably still uses :use but that does not count!

later @ska!

Hey, what’s a good way to not include certain files, directories from your path in resources for making uberjar?

hi @prasincs !

https://github.com/boot-clj/boot/blob/master/doc/boot.task.built-in.md#uber

hi @micha

there are --exclude and --include options

you can also use the sift task between uber and jar to do more complex manipulations

yeah, I tried it a while back.. is there an example? I want to only include say lib/resources/.+/dir in uberjar and ignore the other directories in those directories

— additionally, has someone done any work around deduping depdendencies or removing classes not being used from jars?

boot uber --include 'lib/resources/.+/dir' show -f

the show -f lets you see how the fileset looks after the uber

it'll print a tree to the console

sweet

can I just run show without doing a build?

like a dry run

sure

it just prints a tree of whatever fileset it gets

how does that translate into code?

task-opts! ?

boot uber --include 'lib/resources/.+/dir' show -f == (boot (uber :include #{#"lib/resources/.+/dir"}) (show :fileset true))

ah, thanks @alandipert

np. w/ code is a faster way to develop task stacks in the repl... you can do boot repl and drive the build that way

if/when you update your build.boot, you can reload it in the repl with (load-file "build.boot")

how do I reset the repl setup in this case? or does it do that itself? [I’d like to think boot does the right thing and doesn’t set any globals, just checking :)]

between builds or creating stacks

well, developing on the build.boot itself is kind of a special case

any tasks you define in build.boot are global

but unless you define new ones, no, nothing is retained between boot calls

functions with a bang on the end, like task-options! do mutate state

but most of them, like task-options! in this case, are idempotent

thanks @micha

hello guys, is there a magic way to get the jar files given a dependency inside boot (maybe part of aether.clj?)

@richiardiandrea: https://github.com/boot-clj/boot/blob/master/doc/boot.pod.md#resolve-dependency-jar

that's if you need the jar file for a single dependency

if you want to get all the jar files you can use this one: https://github.com/boot-clj/boot/blob/master/doc/boot.pod.md#resolve-dependency-jars

awesome, I don't even need to map 😄

basically I need to fetch the source jar, if present

yep that will work

so yep, I just need to add _source