Boot new 0.3.1 released — better support for Leiningen templates: works with Hoplon and Chestnut templates now!! https://clojars.org/seancorfield/boot-new

ben@neb:~/workspace/crt > boot -u
#
#Sat Jan 30 17:50:14 PST 2016
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.5.5
#App version: 2.2.0
ben@neb:~/workspace/crt > boot -h
Please download latest Boot binary: 

@micha could you look at the comment I just made to issue #57 on boot-reload? I was testing a fix made by @juhoteperi It is another case of can't delete file. In this case, it is the main.js file. My guess is that the fix worked but took us straight to another error. My guess would be that someone made main.js read only. So, any idea where the new issue should be posted? Such fun.

@richiardiandrea: we may want to rethink the sift task entirely and make a new task called files or something?

to not make breaking changfes

@micha: at the moment in my PR nothing breaks (I tested) are you referring to #401 ?

no, i was thinking about some of the other things you mentioned

improvements to how it works

like separating the "operation" from the include, exclude etc modifiers

ah, well that would be great, but I stopped thinking about it as there was a way to use --include with --add-jar, I just needed a working --add-meta for what I wanted to do

@seancorfield: your blog post was very kind, thanks for that

ah okay

actions in sift are already composable from what I saw

only --add-jar was a bit weird at the beginning, but I can live with that...it would be maybe not good to duplicate sift only for one little UX problem...

@bsima: When I type boot -u it doesn’t report the App version. My BOOT_VERSION=2.5.5.

@micha: NP. I’ll be blogging more about Boot soon. I want to blog about boot-expectations and boot-new as well as some of the stuff we’re doing with Boot to eliminate Ant

ah right, that means you need to download a newer boot executable

2.5.0 and later need a later version of boot.sh

So I guess when it says Please download latest Boot binary: it really means it /cc @bsima

yeah, I'm using nix-env -i boot

but once you get that one you're done, because it is compatible with all previous releases of boot and will be compatible with all future versions of boot, too

trying a manual install now, maybe that will fix it?

hm, i thought nix was all good?

nix usually works no problem for me, maybe I have some weird symlink thing going on

is there something like apt-get update that you can do?

maybe the version in a cache is old or something?

i cleared the cache a bunch of times -- boot isn't the problem

i mean the nix package manager cache, if there is one

looks like it was a nix environment thing. manual install seems to work

I tried nix-collect-garbage and all that stuff ¯\(ツ)/¯

so does nix make it easy to do something like have my package manager cache live on s3?

like if i bring up a server on aws, to install packages from s3 instead of from the internet

looks like the default cache is on S3 anyways https://nixos.org/wiki/FAQ#Should_I_use_http:.2F.2Fhydra.nixos.org.2F_as_a_binary_cache.3F

interesting

this looks like an interesting thing too http://www.aptly.info/

The second in my Boot series: http://seancorfield.github.io/blog/2016/01/30/building-on-boot/ (already added to the wiki).

hey, please check out https://github.com/tulos/boot-criterium - there are no releases yet, have to get my company's Clojars credentials

@dm3: You should not use shared creds clojars but add permission to your own account to push to the group

ok

will do that

still need to login there

Yeah

was wondering if there's a cat task that prints something from a fileset?

Hmh, using creds from ~/.lein/credentials.clj.gpg is quite inconvenient with Yubikey as the file needs to decrypted whenever I run boot, instead of only when I deploy something

Maybe I'll just start using creds from env or from unencrypted file

I have another quick question and possible bug - are boot pods only supposed to work for deps, or should they also work for resource or source path changes?

@weavejester: you mean like creating a pod without local source files or different ones?

@martinklepsch, sorry, I didn’t notice your reply. I mean a pod with different :resource-paths in its environment.

Pods don’t appear to carry across :resource-paths and co. across.

@weavejester: can you paste how you're constructing the pod please?

usually you want to do something like this:

(boot.pod/make-pod (update-in (boot.core/get-env) :dependencies #(conj ...)))

@micha: Sure, give me a sec...

or if you're making the pod from another pod:

(boot.pod/make-pod (update-in boot.pod/env :dependencies #(conj ...)))

that preserves the settings from the parent pod

(sibling pod, technically, but parent-child seems more intuitive)

(def dev-pod
  (boot.pod/make-pod
   (-> (get-env)
       (update :dependencies conj '[medley "0.7.0"])
       (update :resource-paths conj "dev"))))

So the new dependency works, but the new resource path doesn’t.

Also there doesn’t seem to be any code around handling resource paths in the boot.pod namespace, but I might be looking in the wrong place.

you can add to :directories

that's the actual classpath dirs

Yeah, I noticed that - there is code for :directories.

the pods don't really have a useful notion of resource/source paths

they're just classloaders with urls, some of which are maven dep jars and some are directories

this should be documented better

So if I want to have a pod that extends a local directory, would I reference it directly and ignore the fileset, or would I add it to the current fileset?

how do you mean extends?

I mean, if I want a pod to have access to a new classpath directory.

you want to add a directory to the pod's classpath?

Yeah. So I’m experimenting a little with using pods for isolation. Rather than do X or Y directly in the boot environment, keep the boot environment for boot stuff, and put the project stuff in pods.

I know that’s not how Boot is designed to work, I’m just pushing it’s boundaries a little.

that seems to be the intended use case for pods

like you have dependencies that you want to isolate

So that kinda works, except that the :resource-paths aren’t carried across, and exiting the boot REPL will exit both the inner and outer pod. I haven’t figured out a way of exiting just one.

the actual build pipeline needs to run in a single thread though

in the core pod

but the actual work is done in pods

consider the cljs task, for example

that loads all the hairy google closure deps into a pod

then when the task runs it sends the build function in the pod the path to the temp directory that the cljs task created

the idea is that the cljs compiler and goog closure will write to this temp directory

then you add the contents of that directory to the fileset

So it seems like what I’m doing isn’t a million miles away from what boot does already...

What I was trying to do is to have a “dev” pod that I could start a REPL in.

this effectively isolates that messy process from the orderly transformation of one set of files to the next

the main idea is that the cljs compiler for example doesn't have access to the build environment

it runs in a sandbox where it can emit whatever files it wants to without any chance of clobbering anything or messing up the main pod's classpath

and without needing to know anything about the rest of the build process

especially when it comes to where to emit files and so on

this allows you to decouple the tasks

and that decoupling facilitates composition

exactly the same principle as ring middleware

I want to do that for a development environment. So have a pod that represents the development environment that’s isolated. The two problems I had with it is that while you can start a REPL that runs in a pod, you can’t stop a REPL without also stopping the outer one. The second problem is that I couldn’t include new resource paths.

i don't understand the dev environment in a pod setup

why does the dev environment need to be in a separate pod?

like what are you isolating it from?

Well, ideally things like generating production jars, and so forth. It would be nice to be able to test in a dev environment vs. one with only production dependencies. It also feels… more compartmentalised I guess? I mean, boot essentially does the same thing for the filesystem.

hmm

i usually use some variation on the 12-factor app setup

so production vs dev is a change of environment variables

my environment thing will also use system properties, which can be changed at runtime

to override actual env vars

So you use processes for isolation?

no, i can run (boot ...) in the repl

i just set the env vars different ways

then do (boot ...) again

i mean system properties

It's not all that constructive, but I like this idea weavejester is pushing. I kind of wanted to do something similar once - so I could take a pod and create & tear it down when dependencies changes, so there would be about zero dependency clashes. But then I didn't have time to look into that further.

If this is feasible, I feel it could be awesome.

since system properties are JVM-global they are visible to all pods

But doesn’t that pollute the current environment a little? I mean wouldn’t it be more isolated if you could just wrap things in pods that you can dispose of later?

so you can actually do this, by running boot inside itself

https://github.com/micha/multi-module-build/blob/master/build.boot

that's an example of a multi-module build

where you have one build.boot that is run multiple times inside itself

that provides a completely isolated, in-process instance of boot

but the different instances can still communicate as necessary for synchronization, etc

the issue with just wrapping in pods is that one of the main things that boot does is maintain global state in a way that exposes a functional, immutableish interface to the user

but underneath there is a lot of state that needs to be managed by boot

pods do not participate in this, they're just sandboxes where you can run isolated programs

if you want to make an isolated build process then you want to use the boot.App.runBoot() method

like in that example

I don’t think I want to run multiple copies of boot, I more want a way of isolating my project dependencies from my boot ones. So I can say: run a project REPL in this particular context. Not sure if that’s a good idea or not. I might need to rethink.

what are "boot dependencies" that you mention?

boot tasks?

Yep

one of the main characteristics of boot tasks is that they have no transitive dependencies--they do all their work in pods

so there is no benefit to this isolation

because there is nothing to isolate

if you add :scope "test" then your boot dependencies won't be polluting your poms with transitive depdencies, either

I guess the problem I have is that the boot environment is a mix of boot tasks, and the actual project functions as well. I’m coming at this from Leiningen, where the Lein environment and the project environment are separate.

this is how it do it also

So my first instinct with boot is to look at the way it pushes both project and build environments together, and go: can’t I isolate these things?

usually i just deploy my application with boot as the entrypoint

so i can use pods in my application

like instead of doing java -jar project-1.2.3.jar

my upstart script or whatever does boot serve-my-app --port 3000 or something

so my application is sort of frankensteined together with the boot stuff

but it makes it pretty easy to have a good 12 factor design

and easy to run locally for dev etc

also it is nice because it separates the machinery from the application pretty well

I was wondering if I could use pods to get a similar or “good enough" separation between boot and my project that Leiningen provides between lein and my project.

you could run another boot jvm, i suppose

and pass forms via the command line

Well, pods looked nice because they’re faster to start up.

i don't really understand the motivation though

i am slow on the uptake sometimes

I guess the motivation is separation? Maybe I’m thinking about this in the wrong way, but I don’t see why the build environment and the project environment need to be mixed together.

like what is the problem you want to avoid?

The mixing of my build environment with my project environment, mainly. I don’t know whether this is a good goal to have, yet.

i still can't see the problem there, though

i mean leiningen mixes them too, otherwise you couldn't do anything

The difference is, is that when I do “lein repl”, Leiningen isn’t loaded on my classpath. In Boot it is.

The Boot REPL is the same thing as my project REPL.

ok but what is the issue there?

like what damage does that cause that you want to mitigate?

None that can’t be mitigated in other ways… but it feels like a step back? Going from an isolated environment to a mixed one.

i dunno, i don't have any philosophical preference either way, i just want to build things

btw if you want to add dependencies without them appearing in :dependencies in the boot env you can use this function: https://github.com/boot-clj/boot/blob/master/doc/boot.pod.md#add-dependencies

there are drawbacks to that, of course, which is why i recommend using :scope "test" to achieve the same end

I’ll go away and think about this a little more. It might be that this doesn’t matter as much as I think it does. It’s very much a knee-jerk reaction. Isolation good, mixing bad.

But then in my current project REPL, I tend to mix running the app and testing it. Is that the same thing? I’m not sure…

actually there are no drawbacks for using that function to load boot task dependencies

since they have no transitive deps

so there is no problem adding them incrementally

I need to head off now. Thanks for the talk.

for me running the app and testing it are the same really

And for clarifying what pods can and can’t do.

any time! later

It's good to see James playing with boot.

boot is super powerful, the comparison is with gradle as expressiveness, but it is more concise (clojure rulez) and it has macros!

and I don't think gradle has pods 😄

haha or filesets

maybe fileset yes

kind of

not as first class though

in gradle you pass actual named paths to tasks, no?

like tasks are doing things in global named places in the filesystem

vs the fileset where the actual filesystem is hidden from tasks, so tasks only see sandboxed tempdirs

so they don't need to coordinate with each other

yep

I don't like comparisons, but https://docs.gradle.org/current/userguide/working_with_files.html

😄

in any case, don't lose my point, boot is awesome 😄

interesting

it looks like gradle's file thing is referencing real files that exist in the project

is that not correct?

like adding things to the file collection just adds references to real files in your project

like if you add your src directory

for example

if a task replaces like src/Foo.java with a Foo.java that has different contents that will overwrite the actual file in your project src directory, no?

yes it looks like it, there is no abstraction

@micha about option OBJECT ^:! code, is the code executed already by the cli parser before entering the task?

ops sorry the answer is yes (comp eval read-string) 😄

the parser is only used when the option is provided on the command line

there when the option is provided in the repl or in clojure code no parsing is done, but the type checker is used to make sure the option is of the correct type

(the parser guarantees that the option will be of the correct type because it can't emit anything else)

the ^:! meta disables the type checker

so no parsing or type checking will be done there if the user provides the option via clojure instead of the command line

oh

that's why it was not working in the repl

😄

what wasn't working in the repl?

it was about runboot, some things were not working but I am digging digging

hi. could you give a hint how to set up clj- and cljs-paths with boot? i don’t want to put them in the same “src” dir, but rather split paths into “src/clj” and “src/clj”. looks like it’s impossible to set

:source-paths   #{"src/clj" "src/cljs”}

Assert failed: The :source-paths, :resource-paths, and :asset-paths must not overlap

@michal: can you paste the rest of your build.boot please?

you must be setting something else to "src" somewhere

is there any way to see resulting src with repl ?

(boot (show :env true))

are you using bootlaces?

yes

i guess that’s a problem, isn’t it ?

(bootlaces! version :dont-modify-paths? true)

that's what you want, it'll fix your problem

you can see the conflict if you do boot show -e

shouldn’t it be “bootlaces!” (with !) ?

ah ok.

yes, that's correct

yessss… it’s working btw. is it recommended way of how cljs & clj files should be separated?

it doesn't matter except if you prefer it one way or the other really

i usually separate things according to function, like maybe src/frontend and src/backend for example

but it makes no difference to boot, cause it all ends up in the same place

there isn't any problem with mixing clj and cljs source files in the same directory if you want

i see. ok, so I will try with clj/cljs directories layout then. thanks for your help. it saved me few more hours

np! 🍻