Fork me on GitHub
#boot
<
2016-01-31
>
seancorfield01:01:26

Boot new 0.3.1 released — better support for Leiningen templates: works with Hoplon and Chestnut templates now!! https://clojars.org/seancorfield/boot-new

bsima01:01:23

[email protected]:~/workspace/crt > boot -u
#
#Sat Jan 30 17:50:14 PST 2016
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.5.5
#App version: 2.2.0
[email protected]:~/workspace/crt > boot -h
Please download latest Boot binary: 
anybody know why this is happening?

laforge4901:01:28

@micha could you look at the comment I just made to issue #57 on boot-reload? I was testing a fix made by @juhoteperi It is another case of can't delete file. In this case, it is the main.js file. My guess is that the fix worked but took us straight to another error. My guess would be that someone made main.js read only. So, any idea where the new issue should be posted? Such fun. simple_smile

micha02:01:31

@richiardiandrea: we may want to rethink the sift task entirely and make a new task called files or something?

micha02:01:40

to not make breaking changfes

richiardiandrea02:01:23

@micha: at the moment in my PR nothing breaks (I tested) are you referring to #401 ?

micha02:01:41

no, i was thinking about some of the other things you mentioned

micha02:01:51

improvements to how it works

micha02:01:13

like separating the "operation" from the include, exclude etc modifiers

richiardiandrea02:01:34

ah, well that would be great, but I stopped thinking about it as there was a way to use --include with --add-jar, I just needed a working --add-meta for what I wanted to do

micha02:01:37

@seancorfield: your blog post was very kind, thanks for that simple_smile

richiardiandrea02:01:13

actions in sift are already composable from what I saw

richiardiandrea02:01:57

only --add-jar was a bit weird at the beginning, but I can live with that...it would be maybe not good to duplicate sift only for one little UX problem...

seancorfield02:01:03

@bsima: When I type boot -u it doesn’t report the App version. My BOOT_VERSION=2.5.5.

seancorfield02:01:10

@micha: NP. I’ll be blogging more about Boot soon. I want to blog about boot-expectations and boot-new as well as some of the stuff we’re doing with Boot to eliminate Ant simple_smile

micha02:01:11

ah right, that means you need to download a newer boot executable simple_smile

micha02:01:40

2.5.0 and later need a later version of boot.sh

seancorfield02:01:50

So I guess when it says Please download latest Boot binary: it really means it simple_smile /cc @bsima

bsima02:01:14

simple_smile yeah, I'm using nix-env -i boot

micha02:01:16

but once you get that one you're done, because it is compatible with all previous releases of boot and will be compatible with all future versions of boot, too

bsima02:01:24

trying a manual install now, maybe that will fix it?

micha02:01:42

hm, i thought nix was all good?

bsima02:01:04

nix usually works no problem for me, maybe I have some weird symlink thing going on

micha02:01:23

is there something like apt-get update that you can do?

micha02:01:35

maybe the version in a cache is old or something?

bsima02:01:02

i cleared the cache a bunch of times -- boot isn't the problem

micha02:01:18

i mean the nix package manager cache, if there is one

bsima02:01:24

looks like it was a nix environment thing. manual install seems to work

bsima02:01:49

I tried nix-collect-garbage and all that stuff ¯\(ツ)

micha02:01:23

so does nix make it easy to do something like have my package manager cache live on s3?

micha02:01:00

like if i bring up a server on aws, to install packages from s3 instead of from the internet

micha02:01:14

interesting

micha02:01:38

this looks like an interesting thing too http://www.aptly.info/

seancorfield07:01:57

The second in my Boot series: http://seancorfield.github.io/blog/2016/01/30/building-on-boot/ (already added to the wiki).

dm312:01:23

hey, please check out https://github.com/tulos/boot-criterium - there are no releases yet, have to get my company's Clojars credentials

juhoteperi12:01:30

@dm3: You should not use shared creds clojars but add permission to your own account to push to the group

dm312:01:40

will do that

dm312:01:50

still need to login there simple_smile

dm313:01:20

was wondering if there's a cat task that prints something from a fileset?

juhoteperi13:01:36

Hmh, using creds from ~/.lein/credentials.clj.gpg is quite inconvenient with Yubikey as the file needs to decrypted whenever I run boot, instead of only when I deploy something

juhoteperi13:01:39

Maybe I'll just start using creds from env or from unencrypted file

weavejester14:01:22

I have another quick question and possible bug simple_smile - are boot pods only supposed to work for deps, or should they also work for resource or source path changes?

martinklepsch15:01:08

@weavejester: you mean like creating a pod without local source files or different ones?

weavejester17:01:00

@martinklepsch, sorry, I didn’t notice your reply. I mean a pod with different :resource-paths in its environment.

weavejester17:01:53

Pods don’t appear to carry across :resource-paths and co. across.

micha17:01:15

@weavejester: can you paste how you're constructing the pod please?

micha17:01:24

usually you want to do something like this:

micha17:01:50

(boot.pod/make-pod (update-in (boot.core/get-env) :dependencies #(conj ...)))

weavejester17:01:01

@micha: Sure, give me a sec...

micha17:01:08

or if you're making the pod from another pod:

micha17:01:33

(boot.pod/make-pod (update-in boot.pod/env :dependencies #(conj ...)))

micha17:01:50

that preserves the settings from the parent pod

micha17:01:05

(sibling pod, technically, but parent-child seems more intuitive)

weavejester17:01:23

(def dev-pod
  (boot.pod/make-pod
   (-> (get-env)
       (update :dependencies conj '[medley "0.7.0"])
       (update :resource-paths conj "dev"))))

weavejester17:01:45

So the new dependency works, but the new resource path doesn’t.

weavejester17:01:16

Also there doesn’t seem to be any code around handling resource paths in the boot.pod namespace, but I might be looking in the wrong place.

micha17:01:29

you can add to :directories

micha17:01:43

that's the actual classpath dirs

weavejester17:01:48

Yeah, I noticed that - there is code for :directories.

micha17:01:17

the pods don't really have a useful notion of resource/source paths

micha17:01:42

they're just classloaders with urls, some of which are maven dep jars and some are directories

micha17:01:20

this should be documented better

weavejester17:01:41

So if I want to have a pod that extends a local directory, would I reference it directly and ignore the fileset, or would I add it to the current fileset?

micha17:01:59

how do you mean extends?

weavejester17:01:21

I mean, if I want a pod to have access to a new classpath directory.

micha17:01:03

you want to add a directory to the pod's classpath?

weavejester17:01:02

Yeah. So I’m experimenting a little with using pods for isolation. Rather than do X or Y directly in the boot environment, keep the boot environment for boot stuff, and put the project stuff in pods.

weavejester17:01:28

I know that’s not how Boot is designed to work, I’m just pushing it’s boundaries a little. simple_smile

micha17:01:55

that seems to be the intended use case for pods

micha17:01:04

like you have dependencies that you want to isolate

weavejester17:01:12

So that kinda works, except that the :resource-paths aren’t carried across, and exiting the boot REPL will exit both the inner and outer pod. I haven’t figured out a way of exiting just one.

micha17:01:46

the actual build pipeline needs to run in a single thread though

micha17:01:49

in the core pod

micha17:01:04

but the actual work is done in pods

micha17:01:17

consider the cljs task, for example

micha17:01:36

that loads all the hairy google closure deps into a pod

micha17:01:20

then when the task runs it sends the build function in the pod the path to the temp directory that the cljs task created

micha17:01:39

the idea is that the cljs compiler and goog closure will write to this temp directory

micha17:01:52

then you add the contents of that directory to the fileset

weavejester17:01:57

So it seems like what I’m doing isn’t a million miles away from what boot does already...

weavejester17:01:23

What I was trying to do is to have a “dev” pod that I could start a REPL in.

micha17:01:25

this effectively isolates that messy process from the orderly transformation of one set of files to the next

micha17:01:53

the main idea is that the cljs compiler for example doesn't have access to the build environment

micha17:01:20

it runs in a sandbox where it can emit whatever files it wants to without any chance of clobbering anything or messing up the main pod's classpath

micha17:01:36

and without needing to know anything about the rest of the build process

micha17:01:50

especially when it comes to where to emit files and so on

micha17:01:29

this allows you to decouple the tasks

micha17:01:38

and that decoupling facilitates composition

micha17:01:05

exactly the same principle as ring middleware

weavejester17:01:41

I want to do that for a development environment. So have a pod that represents the development environment that’s isolated. The two problems I had with it is that while you can start a REPL that runs in a pod, you can’t stop a REPL without also stopping the outer one. The second problem is that I couldn’t include new resource paths.

micha17:01:08

i don't understand the dev environment in a pod setup

micha17:01:29

why does the dev environment need to be in a separate pod?

micha17:01:51

like what are you isolating it from?

weavejester18:01:32

Well, ideally things like generating production jars, and so forth. It would be nice to be able to test in a dev environment vs. one with only production dependencies. It also feels… more compartmentalised I guess? I mean, boot essentially does the same thing for the filesystem.

micha18:01:13

i usually use some variation on the 12-factor app setup

micha18:01:25

so production vs dev is a change of environment variables

micha18:01:34

my environment thing will also use system properties, which can be changed at runtime

micha18:01:43

to override actual env vars

weavejester18:01:44

So you use processes for isolation?

micha18:01:59

no, i can run (boot ...) in the repl

micha18:01:08

i just set the env vars different ways

micha18:01:15

then do (boot ...) again

micha18:01:29

i mean system properties

jaen18:01:39

It's not all that constructive, but I like this idea weavejester is pushing. I kind of wanted to do something similar once - so I could take a pod and create & tear it down when dependencies changes, so there would be about zero dependency clashes. But then I didn't have time to look into that further.

jaen18:01:46

If this is feasible, I feel it could be awesome.

micha18:01:54

since system properties are JVM-global they are visible to all pods

weavejester18:01:11

But doesn’t that pollute the current environment a little? I mean wouldn’t it be more isolated if you could just wrap things in pods that you can dispose of later?

micha18:01:38

so you can actually do this, by running boot inside itself

micha18:01:03

that's an example of a multi-module build

micha18:01:15

where you have one build.boot that is run multiple times inside itself

micha18:01:36

that provides a completely isolated, in-process instance of boot

micha18:01:50

but the different instances can still communicate as necessary for synchronization, etc

micha18:01:08

the issue with just wrapping in pods is that one of the main things that boot does is maintain global state in a way that exposes a functional, immutableish interface to the user

micha18:01:19

but underneath there is a lot of state that needs to be managed by boot

micha18:01:00

pods do not participate in this, they're just sandboxes where you can run isolated programs

micha18:01:33

if you want to make an isolated build process then you want to use the boot.App.runBoot() method

micha18:01:37

like in that example

weavejester18:01:59

I don’t think I want to run multiple copies of boot, I more want a way of isolating my project dependencies from my boot ones. So I can say: run a project REPL in this particular context. Not sure if that’s a good idea or not. I might need to rethink.

micha18:01:20

what are "boot dependencies" that you mention?

micha18:01:24

boot tasks?

micha18:01:14

one of the main characteristics of boot tasks is that they have no transitive dependencies--they do all their work in pods

micha18:01:26

so there is no benefit to this isolation

micha18:01:32

because there is nothing to isolate

micha18:01:07

if you add :scope "test" then your boot dependencies won't be polluting your poms with transitive depdencies, either

weavejester18:01:59

I guess the problem I have is that the boot environment is a mix of boot tasks, and the actual project functions as well. I’m coming at this from Leiningen, where the Lein environment and the project environment are separate.

micha18:01:25

this is how it do it also

weavejester18:01:35

So my first instinct with boot is to look at the way it pushes both project and build environments together, and go: can’t I isolate these things?

micha18:01:37

usually i just deploy my application with boot as the entrypoint

micha18:01:44

so i can use pods in my application

micha18:01:01

like instead of doing java -jar project-1.2.3.jar

micha18:01:25

my upstart script or whatever does boot serve-my-app --port 3000 or something

micha18:01:00

so my application is sort of frankensteined together with the boot stuff

micha18:01:28

but it makes it pretty easy to have a good 12 factor design

micha18:01:41

and easy to run locally for dev etc

micha18:01:15

also it is nice because it separates the machinery from the application pretty well

weavejester18:01:58

I was wondering if I could use pods to get a similar or “good enough" separation between boot and my project that Leiningen provides between lein and my project.

micha18:01:16

you could run another boot jvm, i suppose

micha18:01:32

and pass forms via the command line

weavejester18:01:52

Well, pods looked nice because they’re faster to start up. simple_smile

micha18:01:55

i don't really understand the motivation though

micha18:01:11

i am slow on the uptake sometimes

weavejester18:01:39

I guess the motivation is separation? Maybe I’m thinking about this in the wrong way, but I don’t see why the build environment and the project environment need to be mixed together.

micha18:01:42

like what is the problem you want to avoid?

weavejester18:01:24

The mixing of my build environment with my project environment, mainly. I don’t know whether this is a good goal to have, yet.

micha18:01:39

i still can't see the problem there, though

micha18:01:12

i mean leiningen mixes them too, otherwise you couldn't do anything

weavejester18:01:52

The difference is, is that when I do “lein repl”, Leiningen isn’t loaded on my classpath. In Boot it is.

weavejester18:01:05

The Boot REPL is the same thing as my project REPL.

micha18:01:11

ok but what is the issue there?

micha18:01:21

like what damage does that cause that you want to mitigate?

weavejester18:01:16

None that can’t be mitigated in other ways… but it feels like a step back? Going from an isolated environment to a mixed one.

micha18:01:51

i dunno, i don't have any philosophical preference either way, i just want to build things

micha18:01:09

btw if you want to add dependencies without them appearing in :dependencies in the boot env you can use this function: https://github.com/boot-clj/boot/blob/master/doc/boot.pod.md#add-dependencies

micha18:01:07

there are drawbacks to that, of course, which is why i recommend using :scope "test" to achieve the same end

weavejester18:01:16

I’ll go away and think about this a little more. It might be that this doesn’t matter as much as I think it does. It’s very much a knee-jerk reaction. Isolation good, mixing bad. simple_smile

weavejester18:01:58

But then in my current project REPL, I tend to mix running the app and testing it. Is that the same thing? I’m not sure…

micha18:01:06

actually there are no drawbacks for using that function to load boot task dependencies

micha18:01:12

since they have no transitive deps

micha18:01:27

so there is no problem adding them incrementally

weavejester18:01:41

I need to head off now. Thanks for the talk.

micha18:01:49

for me running the app and testing it are the same really

weavejester18:01:50

And for clarifying what pods can and can’t do.

micha18:01:58

any time! later simple_smile

danielsz18:01:18

It's good to see James playing with boot.

richiardiandrea18:01:33

boot is super powerful, the comparison is with gradle as expressiveness, but it is more concise (clojure rulez) and it has macros!

richiardiandrea18:01:53

and I don't think gradle has pods 😄

micha18:01:38

haha or filesets simple_smile

richiardiandrea18:01:49

maybe fileset yes

richiardiandrea18:01:10

not as first class though

micha18:01:49

in gradle you pass actual named paths to tasks, no?

micha18:01:16

like tasks are doing things in global named places in the filesystem

micha18:01:44

vs the fileset where the actual filesystem is hidden from tasks, so tasks only see sandboxed tempdirs

micha18:01:02

so they don't need to coordinate with each other

richiardiandrea19:01:26

in any case, don't lose my point, boot is awesome 😄

micha19:01:46

interesting

micha19:01:08

it looks like gradle's file thing is referencing real files that exist in the project

micha19:01:12

is that not correct?

micha19:01:38

like adding things to the file collection just adds references to real files in your project

micha19:01:48

like if you add your src directory

micha19:01:50

for example

micha19:01:41

if a task replaces like src/Foo.java with a Foo.java that has different contents that will overwrite the actual file in your project src directory, no?

richiardiandrea19:01:00

yes it looks like it, there is no abstraction

richiardiandrea20:01:45

@micha about option OBJECT ^:! code, is the code executed already by the cli parser before entering the task?

richiardiandrea20:01:26

ops sorry the answer is yes (comp eval read-string) 😄

micha20:01:17

the parser is only used when the option is provided on the command line

micha20:01:59

there when the option is provided in the repl or in clojure code no parsing is done, but the type checker is used to make sure the option is of the correct type

micha20:01:19

(the parser guarantees that the option will be of the correct type because it can't emit anything else)

micha20:01:31

the ^:! meta disables the type checker

micha20:01:55

so no parsing or type checking will be done there if the user provides the option via clojure instead of the command line

richiardiandrea20:01:15

that's why it was not working in the repl

micha20:01:30

what wasn't working in the repl?

richiardiandrea20:01:45

it was about runboot, some things were not working but I am digging digging

michal22:01:10

hi. could you give a hint how to set up clj- and cljs-paths with boot? i don’t want to put them in the same “src” dir, but rather split paths into “src/clj” and “src/clj”. looks like it’s impossible to set

:source-paths   #{"src/clj" "src/cljs”}
. i get a
Assert failed: The :source-paths, :resource-paths, and :asset-paths must not overlap
exception

micha22:01:02

@michal: can you paste the rest of your build.boot please?

micha22:01:12

you must be setting something else to "src" somewhere

michal22:01:44

is there any way to see resulting src with repl ?

micha22:01:08

(boot (show :env true))

micha22:01:37

are you using bootlaces?

michal22:01:02

i guess that’s a problem, isn’t it ? simple_smile

micha22:01:29

(bootlaces! version :dont-modify-paths? true)

micha22:01:39

that's what you want, it'll fix your problem

micha22:01:49

you can see the conflict if you do boot show -e

michal22:01:22

shouldn’t it be “bootlaces!” (with !) ?

micha22:01:35

yes, that's correct

michal22:01:24

yessss… it’s working simple_smile btw. is it recommended way of how cljs & clj files should be separated?

micha22:01:48

it doesn't matter except if you prefer it one way or the other really

micha22:01:42

i usually separate things according to function, like maybe src/frontend and src/backend for example

micha22:01:03

but it makes no difference to boot, cause it all ends up in the same place

micha22:01:55

there isn't any problem with mixing clj and cljs source files in the same directory if you want

michal22:01:13

i see. ok, so I will try with clj/cljs directories layout then. thanks for your help. it saved me few more hours simple_smile