I'm getting a strange crash that occurs every other time I run my build task:

clojure.lang.ExceptionInfo: java.util.concurrent.ExecutionException: java.nio.file.NoSuchFileException: /Users/gar/.boot/cache/tmp/Users/gar/boot/boot-bowdlerize/jetty/1s66/-rcsl8f/hello/styles.clj

if anybody's interested the code is at https://github.com/mobileink/boot-bowdlerize

the boot-bowdlerize repo now has basic doc and a jetty testapp that uses boot-http, compojure, hiccup, and some bower resources.

the jetty testapp is where the crash occurs.

@danielsz: we were looking into system.components.mongo with @mbut and we were wondering why the start method is not idempotent as recommended in https://github.com/stuartsierra/component#idempotence we just had an issue with the open connections, because accidentally our test suite was creating new mongo components on every run and our ~6 workstations collectively exhausted the connection limit every few days. (plus we had some orphaned test suites stuck in the background which were maintaining thousands of open connections by reconnecting to the mongo server even after we restarted the server

(we are preparing a pull request already, btw https://github.com/exicon/system/blob/f90447ece80b1443fa4d1ceabd4751294848f63f/src/system/components/mongo.clj)

Great. I'll be happy to merge. Thanks!

so should we make start and stop idempotent or was there any reason why u didn't make them so?

onetom: Yes, by all means. No reason other than youth and foolishness.

Also, under the same idempotence header there is note about catching all errors on stop

we also have an adi (http://docs.caudate.me/adi/) component on the way, btw which we just tried to fork with https://github.com/vvvvalvalval/datomock thx to @val_waeselynck very exciting; thanks for making this component collection!

Awesome!

@juhoteperi: thanks for the reminder, i forgot about that!

I don't remember if I have ever hit problems caused by stop throwing, but should be good idea still

not sure what should we do in the catch block though... just do a print? none of the other components handle exceptions either 😕

I use https://github.com/MichaelDrogalis/dire in my system :

(require '[dire.core :refer [with-handler!])
(with-handler! #'com.stuartsierra.component/stop
  "catch all component stop exceptions so that the whole system can always shutdown"
  Exception
  (fn [e &args]
    (println "Stop exception ! " e)))

You could also throw the exception.

That would allow the consumer to act on it.

I let it throw for the start. But for the stop I prefer to make sure every component got a chance to stop. Not sure yet what it the best approach.

Unrelated question : what kind of args can I pass to boot defcli (there is only bool there https://github.com/boot-clj/boot/wiki/Scripts#command-line-args ) ? I would like to pass Files/String paths.

(Also, I can't get the example CLI to get a false value).

(defclifn -main
  [b test-bool bool "A test boolean"]
  (println test-bool))

@nha: i think i hit the same issue and discussed it with micha, but there is no way to represent false flags currently (as in a half year ago). isn't the GNU convention applies a --no- prefix for this purpose?

@onetom: I see, that could work. For now I am just trying to understand the defcli macro : in the snippet below, what are the third and fourth elements ?

(defcli -main
  [t              ;; short name
   test           ;; long name
   WHATISTHIS     ;; ?
   str            ;; parse fn ? (where can I get a list ?)
   "Some doc   "  ;; docstring
   ]
  )

@nha: there is documentation here: https://github.com/boot-clj/boot/wiki/Task-Options-DSL

specifically: https://github.com/boot-clj/boot/wiki/Task-Options-DSL#anatomy

@micha perfect, thanks

(I already used tools.cli though so this will be for next time - added a link in the wiki also)

Seeking feedback/collaboration on https://github.com/mobileink/boot-bowdlerize, which basically generates runtime config namespaces from buildtime config namespaces, with support for bower. This tasklib resulted from my pondering the advice from @micha and @alandipert regarding management and partitioning of config data in https://github.com/migae/boot-gae . I haven't reorganized the latter, but needed something like boot-bowdlerize for another project. I have one good use case for it (not yet illustrated, the lib is still under dev); I'm curious if anybody else would find this useful. Or is it overkill?

Quick question - how do you create a file in the temporary directory created by tmp-dir!? Usual Clojure file manipulation suspects or are there some helpers I'm missing (at least I can't see them in boot.core)?

it's a normal directory, only managed by boot

so you can use normal Java tools

meaning , etc

I see; I figured maybe there may be something akin to tmp-dir! for tmp files.

But yep, make-parents work as it should; thanks!

boot operates at the directory level for temp files

you can do whatever you want inside the direcotry tmp-dir! returns

boot will just delete everything when it cleans up

Right, that makes sense; I was just kinda confused by parent directories not being created, but that's not boot's fault.

fyi boot-bowdlerize now has a sample bowdlerized Polymer testapp.

I'd like to give a boot talk (mostly intro, explaining core concepts and "why") at the Philadelphia Clojure Meetup (a.k.a. Clojadelphia), it's coming in a couple of weeks, but I am a bit swamped with work and won't have time to make solid slides. I'll mostly do coding examples, but does anyone have any ready Boot materials that is ok for me to show for the intro? I'll of course mention the author and give all the proper credit.

@tolitius: I have somewhat dated slides: http://www.slideshare.net/juhoteperi/clojutre-build-tooling-with-boot-42005279

@juhoteperi: thanks! I'll pick out some slides from there, especially the "why" part, since those problems with lein are fundamental and still are there

what's the proper way to reload the build.boot file when in the repl?

(load-file "build.boot")

ooh, forgot about load-file

thanks

say I store in the initial boot set-env! some info that I pass to tasks; is there a way to make those tasks "see" changes to that info in the env while in the watch pipeline?

i don't recommend doing that

because it breaks the pipeline model in some ways

tasks should be able to participate in the configuration process

which they can't do if the configuration is in a global var

because global vars are not immutable

if you put your configuration info in the fileset though, then it's open rather than closed

and tasks can participate in the configuration process

because the fileset is immutable

so tasks only see what the tasks preceding them in the pipeline did

that's better than having all the tasks banging on shared mutable state

OK, so I move that config from set-env! in build.boot to a config.edn file or something?

so for tl;dr; i would recommend making a file with a unique extension, yes

that way other tasks can modify that to help automate things

I thought it was OK that say a complex dev task knows how to extract info from the env and pass it to different tasks as appropriate though

you can do it, but you are then just storing stuff in a data bag

I don't really need tasks to modify that, but take it as an input

so you lose the ability for tasks to modify that

well that means you need to specify everything explicitly

if all tasks worked that way boot would be just a huge configuration map

without the power to compose tasks

sure, I see the point in that

like the boot-cljs, cljs-repl, and cljs-reload tasks would need to be combined into one monolithic task

yeah, that is the classic example

and for hoplon i'd need to combine all four

but there are plenty of tasks we have that don't actually modify the fileset

testing tasks for instance

they also get their configuration from command line args

which doesn't change in the pipeline

that's the key thing

they don't have a data bag of config map

ok so say I have a testing task that takes which namespaces to test as an arg...

And I'd like to be able to change that set, while developing, without restarting the pipeline

that's a perfect place to use a spec.test.edn

or something like that

a specification

that informs your thing how to configure itself

because if you're already doing fancy stuff at runtime you are going to want to make tasks to automate that

even if you don't need to now, you will soon enough

another option is to add metadata to namespaces

things like that

store the information in the environment itself, not in a map

you can alter-meta on namespaces

ok, i think the spec file is an approach we could use

if you just use a regular var you can add a watch to it

would there be any philosophical argument against generating that spec file from a map in build.boot itself?

i've been meaning to do that with the cljs.edn files for a while now

yeah that's the whole point!

i never create those files manually

i ahve tools that do it automatically

gotcha

like framework level things

for example hoplon automatically generates both the html file and the cljs.edn file

you just have a cljs program, and hoplon can generate all the stuff you neeed to make it a webapp

and can you modify the hoplon task while it's in the watch pipeline, or do you need to restart?

like if some arg to that task should change

the task is reading the file

so if the file changes it will see a new thing

reading a file then generating other files?

not following

you said it generates the cljs.edn files

yes, based on other files in the fileset

it looks for *.hl files

those are the cljs programs

right, ok, so that's not even passed in via params

each one gets a html file generated for it, and a .cljs.edn file

it just knows where to look in the fileset

yeah it scans

the task could be looking in other places

the point is that it's generating the cljs.edn files

so should it be possible to switch boot-cljs from :none to :advanced without restarting the pipeline?

in theory yes

if I had a wrapper task that reads some file to determine what :optimizations to use

that still depends on how the cljs compiler handles that

it might mean trashing the compiler state

ok yeah I've never actually tried that craziness, just using it as an example of a task that currently requires an arg for configuration

the main idea is that if the state of the build is contained inthe fileset then you can do all things

once you start storing state in mutable places in the clojure runtime you lose that ability

ok

i think i see the error of my ways

the crucial thing the fileset provides is the automatic rewinding to the initial state each time the pipeline runs

that means that tasks can progressively enhance the configuration

without running into the problems of unmanaged shared state

@alandipert: great, thanks!

thanks for setting me straight once again @micha, think I know how I want to proceed now

🍻 !!

👽

anybody else out there using boot to work with webjars? i didn't even know there was such a thing until @juhoteperi clued me in this morning, so i've whipped up a few tasks, e.g. webjars (in my world) are build-time only, so boot show -d won't show them, so i've got a show task in boot-bowdlerize to show build-time webjar deps. boot is so cool. anyway, i'm interested in knowing how people deal with webjars in clojure apps.

a related question: any particular reason boot.aether is not documented and exposed as an api? i'm using boot.aether/dep-tree, there must be other useful stuff in there.

Hey guys, what is wrong with:

(pod/with-eval-in pod
        (boot.util/info "%s" (str (:source-paths boot.pod/env)))

looks good to me...?

you may need to require the namespaces?

in the pod

I get

Caused by java.lang.IllegalArgumentException
   Arguments must be either all strings or all fns

you mean boot.pod?

that exception is coming from the boot function i think

i think the pod expression is okay, the problem is elsewhere where you call boot

ok, digging...

I had forgotten with-pass-thru 😞

and I never remember what I need to do to print info in a pod..

(pod/with-eval-in @pod
          (require 'boot.pod)
          (require 'boot.util)
          (reset! boot.util/*verbosity* 3)
          (boot.util/info "Hey %s" boot.pod/env))

ah @micha wait, the info is printed, but not in the repl

it works on the command line

weird, could be some nrepl weirdness with threads

yeah

@micha so if I have my files on :source-paths in a pod, shouldn't I be able to launch a method on them? In my case the -main?

pods look at :directories, but yes

require the namespace, etc

depending on how you require you may need resolve too

I have not changed :directories but outside the pod I did not have the same source path

pods don't look at :source-paths or :resource-paths etc

ok

they just care about the classpath, which is :directories

that should include both ;source and :resource-paths

but if I create a pod by modifying the old env, and adding my source-paths are :directories changing accordingly?

from what I see the answer is not...at least here

(let [options (options :generator)
          pod-env (merge-with (fn [_ new] new) (get-env) (:env options))] ;; <- brand new source and dependencies
      (let [pod (future (pod/make-pod pod-env))]
        (pod/with-eval-in @pod
          (boot.util/info "Directories %s\n" (:directories boot.pod/env))
          (require 'cljs-api.generator)
          (cljs-api.generator/-main))
        (pod/destroy-pod @pod)))

yeah that won't work

pods don't care about :source-paths

so that will be ignored

:source-paths is just a thing in the core pod

and the classpath is really defined by :directories there too

ah, I expected make-pod to take care of that too

pods i think should be lower level things

so if I point my :directories to my new source path it should work..

i'm not sure it's a good idea to mock core pod behavior for them

makes things less direct

the goal is to obtain classpath isolation so maybe I am not Doing It Right 😄

:directories points to a set of paths to directories on the filesystem

if you add a path to another direcotry then that one will be on the classpath when you make the pod

if you're making a pod the classpath will be isolated, for sure

great...so what am I missing, add-classpath with my new source path?

if it's in the :directories set that you pass to make-pod then it will be on the classpath

(boot.pod/add-classpath "src/clj") works too

java.lang.IllegalArgumentException: No implementation of method: :make-reader of protocol: #' found for class: nil
     clojure.core/-cache-protocol-fn                  core_deftype.clj:  554
                                            io.clj:   69
                                          io.clj:  102
                                 ...                                        
                  clojure.core/apply                          core.clj:  632
                  clojure.core/slurp                          core.clj: 6653
                                 ...                                        
cljs-api.generator/load-cljs-api-edn                     generator.clj:  176
            cljs-api.generator/-main                     generator.clj:  207

I can launch it now at least 😄

that's what you see when the path doesn't point to a directory

actually you're calling slurp on nil

yes, it is a problem in my code now I think

actually this guys reads a file that I need to add to the class path as well

Cljs api dumped in src/cljs/cljs_repl_web/cljs_api.cljs
Generated docs for 120 symbols

sweet!

thank you a lot 😉

the dreaded "No reader function for tag object" is giving me fits. again. this time, I do let [tgt (boot/tmp-dir!)] at the beginning of my task. then as soon as i enter boot/with-pre-wrap` i do (boot/empty-dir! tgt). but then, inside my pod, any reference to ~tgt` gives me the dreaded exception. i'm trying to use pod/unpack-jar with no luck. a string dest didn't work so I'm trying to make a File object on a tmp-dir.

@mobileink: only forms that can be printed with pr-str and read via read-string can pass betwen pods

that's why you see that exception

you want to pass in the string path to the java.io.File object

not the object itself

i.e. pass ~(.getPath tgt) instead of ~tgt

the test it must pass is (= tgt (read-string (pr-str tgt)))

thanks, let [destdir (str ~(.getPath tgt) "/" ~base)] in the pod did the trick. That definitely deserves a branch in the boot-fails project.

haha

out of curiosity, what's the reason for this restriction?

it's the way clojure works, interfaces and classes are created dynamically at runtime

so two clojure runtimes can't understand each other's clojure things

right, forgot pods are runtimes

i have an experimental method that removes this restriction mostly

uses reflection heavily though so is slower than pr-str

fyi i just copied this to a new item on the Troubleshooting page in the wiki. Not the first time I've encountered it and I'm pretty sure it won't be the last.

thanks.

👍 thanks!

I think I have also added it to the docstring PR

FWIW I'm beginning to see that filesets are not the only killer feature of boot. pods are nutty cool. I wonder, can competing java-based build tools do this kinda stuff? I'm looking at you, gradle. stuff like support isolated class paths .