also @micha, is adding meta to TmpFile as simple as with-meta ?

actually you can just assoc

that's how the --add-meta thing works

beacuse it's not really metadata to the program, it's metadata from the user's perspective

yes I was actually checking that in boot.core

but that metadata should still be considered in equality tests for instance

i mean the fileset metadata added by the user

you also dissoc [:dir :path :id and :time]

those are the fields of the TmpFile record type

hm, we added a couple of fields i think

yes, I might actually just call boot.core/add-meta that does the right thing 😄

haha 👍

last thing @micha, how can I explode a fileset in its java.io.Files?

because add-meta needs filepaths

can I just take :tree ?

the java.io.Files?

i don't think anything cares about the file paths, only the relative paths in the fileset

the :path

also the keys of the :tree map

but about the file inside the fileset...is :tree what is needed for the TmpFiles ?

ok

tnx!

thank you!

(I need this to work asap ahahah!)

is there any template like tenzing but with a backend? (ideally saapas as a template)

@bensu: don’t know of one - but with boot new coming from @seancorfield - perhaps we can get one put together. What stack would you start with? Same as saapas?

@donmullen: thanks. saapas would be nice

if it doesn't exist, in the future I'll probably make saapas into a template

I’d be thrilled to see a something/boot-template up on Clojars as something to point to for others!

Soooo @micha sorry to bother again, but is the following correct:

:tree
 {"replumb/boot_cljs_src.clj"
  {:dir
   #object[java.io.File 0x5dff50b "~/mylongdir/boot-cljs-src/cwn/-x24pa9"],
   :bdir
   #object[java.io.File 0x73faa25d "~/mylongdir/boot-cljs-src/cwn/-7kd6cy"],
   :path "replumb/boot_cljs_src.clj",
   :id "13a18d07a582fd99407bf48fa488f89b.1453772310000",
   :hash "13a18d07a582fd99407bf48fa488f89b",
   :time 1453772310000,
   :boot-cljs-src-tag true}, <- added metadata
  ....  

that looks right

the value should be a record, right?

of type TmpFile

yes......but with-meta returns nil because it looks like it checks (meta x) (in https://github.com/boot-clj/boot/blob/master/boot/core/src/boot/task_helpers.clj#L63)

ah that could be the issue, right?

I can fix that, it should be a normal get, I will try, but I will break things 😄

hahaa 👍

sooo, I investigated what is passed inside sift-meta and it looks like it is just a string, which is the file path, no TmpFile instance...

boot ... sift --with-meta foo

that should remove all paths (and associated TmpFiles) from the fileset that don't have the following condition:

Look in here, https://github.com/boot-clj/boot/blob/master/boot/core/src/boot/task_helpers.clj#L90

(contains? (get-in fileset [% :foo]))

the reducer

where % is the path

is passing just k not the rest (`:dir ...`)

to match?

you can confirm it by printing inside sift-meta

(defn- sift-meta
  [invert? kws]
  (->> (map #(fn [x] (do (clojure.pprint/pprint x)
                        (contains? (meta x) %))) kws)
       (apply some-fn)
       (comp (if invert? not identity))))

yeah that doesn't look like it's going to work

so I will change sift-filter and the other fns...shouldn't be that hard to keep the same behavior

sift-match can accept k and v where v is a TmpFile

sweet, sounds great

sorry match? but ok it should work

@jaen: are you using the ecs-cli? I'm trying to follow your panda5 example, but ecs-cli is actually so limited... it doesn't support build

@jethroksy: I've used AWS before, but not from the CLI. And that university project was only deployed on Heroku, then Openshift and then Digital Ocean, though I don't think deployment on EC2 would be all that different from DO.

ah... I think i'm gonna end up using just a raw EC2 instance

Yeah, that'd be the easiest, I think.

Or you can create a docker image, put it in a repo and then create a scaling group with run configurations pulling that image and running that.

Done it with a Ruby deployment before, it worked quite okay.

The downside is your app has to be stateless, so you should do things like uploading files to S3 and such.

If I've linked you to this - https://gitlab.com/jaen/panda5 - instead of the github repo (I don't remember) then this has docker image creation, so the only part left would be to figure out the scaling groups thing.

This sounds sensible for example - http://www.ybrikman.com/writing/2015/11/11/running-docker-aws-ground-up/

Though it might be an overkill for just a simple app.

Let's say boot-http wants to expose the port it choose randomly to other tasks in the pipeline: what is the best way to do this? I'm thinking of adding a file to the fileset but also technically it's possible to just (assoc fileset :http-port p) - other options?

This is a discussion on tasks and meta data: http://hoplon.discoursehosting.net/t/best-practices-for-creating-extensible-tasks/358/5

I found the following quote there: > It's also possible to assoc directly onto the fileset object itself for project-level metadata, if desired.

As the boot-http task will also serve files that are created downstream, I see no real relation to TmpFiles here.

So this is why I would consider it "project-level metadata"

I haven't seen code doing this before but given the discussion there probably its fine. Sorry for the confusion then I guess :) @codebeige

I was also looking for any examples before, could not find one, either.

So it is not really "best practice" yet.

One problem I could see is in the context of a watch task that creates new filesets which might drop this data (there is some merging done there not sure if outside of :tree as well)

In any case probably good for now :)

One thing I noticed: when using a threading macro non-forms will automatically wrapped in parens so you don't have to do that.

Yes, this is true. I tend to keep them around, if there also a form that takes options and has parens. I like the alignment better. But this is purely personal taste.

Hello! Anyone using emacs flycheck with boot repl? It works fine with lein repl.

anyone using boot to do electron stuff w/ cljs?

i'm curious in general to know what the state of boot/node/cljs is

@alandipert: I did some electron stuff — anything in particular you'd like to know?

@alandipert: https://github.com/martinklepsch/electron-and-clojurescript

fairly straightforward really

@juhoteperi: just stumbled upon this: https://github.com/jafingerhut/dolly — might be handy for the namespace importing in boot

I'm doing some housecleaning with boot-http PR's and I'm wondering if anyone has any opinions re: these - https://github.com/pandeiro/boot-http/pull/32 & https://github.com/pandeiro/boot-http/pull/35

Both use-cases are things that I had previously considered solvable by just writing a custom handler, which has always been supported by boot-http

BTW @codebeige thanks for your PR's; I'll merge those as soon as I decide what to do with these two ^

@pandeiro: maybe instead of adding options the README could list these scenarios with fitting handlers? They both seem useful but also add bloat that might be better solved with documentation.

The README is also getting bloated @martinklepsch ! 😄

😄

Then a "Recipes" wiki page or so?

I think what I'm saying is just adding more and more options will exponentially increase the number of combinations and people might expect them to all work seamlessly when they don't necessarily do. Helping people understand how they can write their own simple ring handlers to accomodate their needs might better for everyone

Yeah I definitely see your point there.

I guess I'm trying to assess how common these scenarios really are.

also testing that all things on their own work becomes a lot more work

Yes for sure. Already supporting Jetty and HTTPKit means testing almost every PR in two variations at least

At the same time, this lib has always tried to be somewhat of a Swiss Army Knife, solving lots of different HTTP serving use cases

It would be nice if there were some way to poll the community of users in situations like this, affecting the API

Is there a better way to do that than informally, here?

i would say PRs are votes in that case

and then what are the nay votes?

comments on PRs?

your Vision

like the 404 thing... i would probably veto, because that's application behavior i'd rather have in application code than in my build.boot

but otoh, it seems like something a Swiss Army Chainsaw would do

@alandipert: not necessarily if they're writing a frontend that uses pushState?

well they shouldn't do that either

(assuming that is out of question )

but again there's plenty of room for there to be lots of libraries that do what different people want

or for one library to do everything, if the maintainer is up for it

@pandeiro: maybe instead of adding options to the task the lib could come with some sort of contrib namespace where people can contribute more general handlers

and then people could use these for the :handler option

Wow @martinklepsch I think that is a very cool idea, pre-fab handlers one can drop in

this way the option explosion wouldn't be there at least :))

plus there is a clear path for future contributions in similar directions

@alandipert: @martinklepsch : tell me more about NOT using pushState for SPA front-end (static site).

@donmullen: I think the general point is that it muddies the water between the correct "entry point" into an application from an HTTP perspective. I draw most of that observation from conversations with @micha btw

So we’re considering using the technique here : https://stackoverflow.com/questions/16267339/s3-static-website-hosting-route-all-paths-to-index-html

To serve up and support links into the static site — is there a better way?

if you handle all requests through a generic handler that serves the app and place the routing logic inside the app itself (but parsing URL paths instead of hash fragments), there's no need for pushState, is there?

As I understand it, the main objective here is just getting rid of the leading #/ portion of the app's routes?

uri fragments!

@micha: I’m having a dejavu.

haha

@pandeiro: the thought was not to use #/ - and yes handle the routing client side

@micha for hoplon SPAs - you are still recommending #/ ?

Ah so what pushState accomplishes is hijacking the browser refresh to not load the index.html etc again?

(while still maintaining the back button / history)

@pandeiro: that’s my understanding - but I know only enough to be dangerous 😉.

@donmullen: re: hoplon yeah, but the redirects looks workable too (just more complicated)

@pandeiro: why not introduce a namespace to boot-http providing a bunch of more common handlers?

@donmullen: if you're using S3 I might just do a little plug here: https://github.com/confetti-clj/confetti

one problem with hosting things on S3 is how they do redirects, like from http://example.com/foo --> http://example.com/foo/

@martinklepsch: hmmm, yeah I suppose that could be done; but is there any reason to do that instead of a wiki page linking to separate projects with handlers? is it easier this way?

they use 302s and not 301s, which is apparently confusing to google's webcrawler and you supposedly get penalized

@pandeiro: just thinking bundling things up might be easier for users — mostly "discoverability" concern I guess

i made a varnish proxy that sits in front of s3 website at adzerk to address that

@micha: if you have a foo file they will redirect to /foo/?

Another thing I just realized @martinklepsch is that the :handler option takes a function, but that function is not really parameterizable

So some of these (like the forwarding) would need to become a handler factory

Thanks @martinklepsch

:handler (forward {:opts ..}) returning a handler?

Yeah something like that

@martinklepsch: if you have a foo file then no, it won't redirect, but if you have a foo/index.html then it will

but on S3 you could have both, i believe, since there are no real folders

I see. Not something I've had a problem with

Oh right

our SEO guy demanded action lol

haha

what's the best way to find files matching a predicate in the fileset?

use the sift task from my task?

@dm3: (->> fileset boot.core/ls (filter pred?))

there are a number of built-in ones, in boot.core

for example:

(->> fileset core/input-files (core/by-re [#"\.txt$" #"\.sh$"]))

returns a seq of TmpFile objects whose paths end in .sh or .txt

thx

working on a task that wraps criterium

was planning on capturing the result edn as files and then having another report task to format and print to stdout/write html into a file

what do you think @micha?

i think that's a fantastic plan

especially separating the notification part

we've been wanting to explore this with boot-test for a while, too

ok, so I managed to write into the file using the (let [t (core/tmp-dir!), f (io/file t "x")] (spit f content) (core/add-resources fs t))

should I query the tmpfile back from the fileset using the name to add metadata to it?

or is there a better way?

I see there's core/add-meta which expects a map {tmpfile metadata}

@micha how do I find the tmpfile?

sorry, one sec

@dm3: how do you mean "find the tmpfile"?

if I write to a file during the task

and want to call core/add-meta

what is the proper way to construct the argument to add-meta?

(-> fileset (add-resource tmp) (add-meta {"some/path.sh" {:foo 100 :bar 200} "some/other/path.sh" {:foo 666 :bar 777}) ...)

ok, got it

need to get the tmpdir path and create the path to my file

note that the "some/path.sh" is not the path to the actual file

it's the relative path in the fileset

none of the functions that involve filesets ever use the actual path to the real file for any purpose whatsoever

tmpfiles are essentially anonymous

yeah

hm, now I'm stuck constructing a path out of a tmpdir and a file name

the best way, cross platform, is this

how do I get the relative tmpdir path?

(.getPath ( "some" "path.sh"))

i don't understand

get the relative path from what

from the (core/tmp-dir!)

I assumed that the path to the returned file cannot be used

there is no relative path

that just returns an anonymous, boot-managed temporary directory you can use to store files

yes, but how I go from there to "some" in your example?

(let [tmp (tmp-dir!)]
  (io/file tmp "some" "path.sh")
  ...

ah, the tmpdir is the "root"

makes sense, as it takes no arguments to create

I was confused thinking it's nested

yes i'm assuming that later you do something like

(-> fileset (add-resource tmp) ...

thanks

🍻

I’ve been playing around with Boot, and I have a quick question about dependencies...

hi @weavejester ! fire away

Well, typically I want a set of dependencies for development, and a subset of those for production (such as generating an uberjar). In Leiningen I’d use the :dev profile. I’m not certain what the most idiomatic way of doing it in Boot is.

I can make a “dev” task, but it feels clunky, and doesn’t work well with require

the simplest way is to use scoped dependencies, i think

dev deps have :scope "test", which won't be included in uberjars by default

you can specify which scopes go into the uber if you want, too

there is an option on the task to change the default to whatever you want

Scopes are fixed, aren’t they? I can’t create my own scopes? Or can I?

“test” just doesn’t seem right if I’m not testing.

i think it's right, because in maven dev and test are complected

Maybe put your dependencies in a map in an EDN file, read that file, merge the dependencies you want, and then use that to set-env!?

but since you're not going to be pushing your uberjar to maven as a dependency you can invent your own scopes, no problem

By “doesn’t seem right” I mean that it feels messy, even if it works!

i don't see why it's messy

just replace "test" with "dev" in your mind

because that's what maven should have called it anyway

scope test doesn't actually have much to do with actual testing in maven either

Bonus points for being able to easily load new dependencies without killing boot (conflicts notwithstanding) in that way.

it's just a way to specify dependencies that are not to be propagated transitively

but without eliding them from the pom, because they may be useful as a reference

jaen: I could also put the base dependencies in their own def

@weavejester: true, but then you lose the bonus points (the ability to add dependencies on the fly, to be clear).

micha: I generally don’t want them in the pom, though.

you can use the :dependencies option to the pom task to edit what goes in there

but poms for uberjars is a dicey proposition anyway

I guess maybe what I do is (def base-dependencies ‘[[…]]) then

Well, I didn’t just mean uberjars. Jars too.

yeah the only thing to watch out for is if you perform the dependency resolution in two steps you are not guaranteed to get the same dependency graph as if you resolved all dependencies together in one step

because of how aether works it's impossible to guarantee that (it's not transactional)

Well, I thought that for production I’d perhaps use a pod.

yeah pods get all their deps resolved when they're constructed, so that's good

also the things you'd put in :scope test are things like boot tasks

which have no transitive dependencies

so those are totally safe

they do their work in pods if they have dependencies of their own

you could use that pattern to eliminate the problem

do dev work in pods

So maybe I’d (def base-dependencies ‘[[…]]) and then (def dev-dependencies ‘[[…]])

to leave the production classpath pristine

And (set-env! :dependencies (into base-dependencies dev-dependencies))

sure, that would work

Then for production I’d just use base-dependencies

you could make two profile-type tasks, dev and prod or something

Yes, though the problem with that is require.

(deftask dev
  []
  (set-env! :dependencies (into base-dependencies dev-dependencies))
  identity)

(require ‘[adzerk.boot-test :refer :all]) needs to be at the top level otherwise it won’t resolve symbols at compile time.

So requiring tasks is a little problematic.

hm

See the problem? A lot of tasks are dev dependencies.

scopes, i thnk, are the way to go there

much simpler

But then all my development deps find their way into my pom file.

not necessarily

Regardless of whether they’re actually used...

so you def your two things, dev and base deps

with dev deps :scope test

then when you make the pom you do

(task-options!
  pom {:dependencies base-dependencies}
  ...

that will ensure that the pom will only have base dependencies

and the :scope "test" will work with the uber task to make sure those deps aren't included in it

But I could also just use the base-dependencies in the uberjar task as well, right?

well you'd do (set-env! (into base-dependencies dev-dependencies) at the top level of your build.boot there

to avoid compilation problems

the uber task discriminates on scope

when it decides which dependencies to include

(set-env!
  :dependencies (into base-dependencies dev-dependencies))

(task-options!
   pom {:dependencies base-dependencies}
   uberjar {:dependencies base-dependencies})

That would work too, right?

there is no uberjar task, though

and the uber task does not currently support a :dependencies option

@micha In the role of squeaky wheel, I'm even having problems with watch on windows when using a library that contains .cljc files. What repository should I write up the issue on???

although it should

the uber task has :include-scope and :exclude-scope options

It doesn’t? Hm.

Perhaps a pod, then?

@laforge49: if you could open a ticket on boot-clj/boot repo that would be awesome

Sure!

@weavejester: is there anything that breaks if you add :scope "test" to your dev dependencies, which won't end up in the pom?

you could also do a tricky thing, actually

lol i see a way

Hm, maybe not. It just feels a little messy compared to Leiningen’s profiles.

A tricky thing?

(def alldeps (into basedeps devdeps)
(set-env! :dependencies alldeps)
(set-env! :dependencies (vec (set/difference (set alldeps) (set devdeps))))

removing dependencies from the env won't actually remove the jars from the classpath, that's impossible

but yeah, i think scopes are the best way

that's what the abstraction was designed for anyway

the scope is intended to be a way to discriminate between dependencies that are needed at runtime from dependencies needed when compiling or using the artifact as a consumer

which i think perfectly describes what you are trying to achieve also

so scope test here, without any pom :dependencies shenanigans is i think the correct way to configure it

most in line with the rest of the maven infrrastructure

That does make sense, but to my mind the way that Leiningen’s profiles work is better, because each set of dependencies and options is strictly isolated.

https://github.com/boot-clj/boot/issues/397

I guess profiles are the current best option in Boot, but I’ll give this some more thought.

I feel like there should be a better way

if you think of something i'm all easr

Has anyone experimented with running a REPL in a pod before? Or using nREPL middleware to “enter" a pod?

@weavejester: you can use boot.core/launch-nrepl to start repl servers inside pods

show --list-pods will give you a list of all the pods

also the repl task has a --pod/:pod option

do you see a need in the boot.core/by-meta function which would filter TmpFiles based on the metadata?

Ah cool, thanks

it kind of breaks the convention with other by-name/re filters by working on TmpFile instead of java.io.Files

metadata is pretty simple

(->> fileset output-files (filter :foo))

that would select tmpfiles with the :foo metadata

i guess it really should be

(->> fileset output-files (filter :foo) set)

because those functions usually return sets

a by-meta function wouldn't hurt though

for completeness

and people could look at it to see examples of using metadata on tmpfiles

haha so yes

@micha: What does it mean by the name of the pod in the :pod option?

(->> fs ls (core/by-meta [:my/meta #(= (:other/meta %) "test")]))

@weavejester: you can use boot show -l to get the pod names

usually the name is the namespace in which the make-pod function was called

but the name can be set to anything

so doing (boot (show :list-pods true)) you can see the names of the pods

you can't get a repl in the worker or aether pods

well maybe you can actually

Thanks, I see it. Looks like the pod name is “anonymous” if you create an anonymous pod. Could you tell me how I set the name?

(.setName the-pod "doop")

.setName on the pod

yeah it's anonymous if you create it from the repl because there isn't a namespace associated with it sometimes

it tries to inspect the call stack, which is weird in the repl

Perfect, thanks

@martinklepsch: thanks for the electron pointer btw. no specific use in mind, just wanted to survey

Out of curiosity, does destroying a pod also kill off any threads it has? It’s in a separate classloader, right?

@weavejester: yes

micha: Very nice

well it's not really possible to guarantee that all threads will be killed

but it does call shutdown-agents etc

Yeah. In the same JVM there are always limits.

true

great! i was about to implement the above! ^

i’m making a boot project that compiles both clojure and clojurescript in separate tasks. is there a way to make them use different :source-paths? I tried calling set-env! in each task but when I compose them into a single task this doesn’t seem to work.

hi group. does it make any sense to want to have my scss in a jar that could be brought in with checkout task? the idea is that I have this common css that I want to share between related projects. if it's not completely crazy, does anyone have any idea about how to make that happen?

@raywillig: Makes sense, but it requires that scss task reads imports from classpath or copies the files to a temp dir and passes that to scss compiler. Sass4clj does first.

oh cool. i will check this out. so I would use this i/o of boot-sass?

@juhoteperi: ^^

Instead of boot-sassc yes, to ensure confusion, my task is also named boot-sass.

haha, yes. ok i get it. sort of i guess. i suppose there aren't really many naming options when making a task to compile sass in boot