This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-02-14
Channels
- # aatree (1)
- # admin-announcements (1)
- # beginners (12)
- # boot (359)
- # cljs-dev (23)
- # cljsjs (2)
- # cljsrn (1)
- # clojure (53)
- # clojure-czech (5)
- # clojure-madison (1)
- # clojure-russia (15)
- # clojured (8)
- # clojurescript (16)
- # community-development (2)
- # core-async (6)
- # cursive (19)
- # datomic (31)
- # emacs (48)
- # euroclojure (3)
- # hoplon (29)
- # microservices (7)
- # mount (46)
- # off-topic (7)
- # om (11)
- # proton (74)
- # reagent (4)
- # spacemacs (3)
Hi I'm curious as to whether there is a comman to generate a generic boot project template. Similar to 'lein new app'
It’ll get integrated into Boot itself at some point. Micha and I have had brief discussions about that already.
I want to get all the basic functionality in place and well-tested, and incorporate some community feedback, so I can feel happy that it’s "ready" to go into core...
While I have your attention…
What’s your thinking on util/exit-error
vs just throwing an exception?
(specifically in the context of https://github.com/seancorfield/boot-new/issues/11)
The problem (from my point of view) is that throwing an exception that the BootApp catches leads to a horribly ugly experience for end users.
So it feels like there needs to be a distinction between "OMG! I has errorz!" and "I’m sorry, I can’t do that Hal!" in terms of exiting the Boot pipeline with a non-zero status code to the shell.
For example:
(! 502)-> boot -d seancorfield/boot-new new -n funjure
Sorry, names such as clojure or *jure are not allowed.
If you intend to use this name ironically, please set the
BOOT_IRONIC_JURE environment variable and try again.
clojure.lang.ExceptionInfo: boot.App$Exit: 1
data: {:file
"/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/boot.user3176105402114756413.clj",
:line 17}
java.util.concurrent.ExecutionException: boot.App$Exit: 1
boot.App$Exit: 1
boot.new/create/invokeStatic new.clj: 88
boot.new/create new.clj: 80
boot.new/eval1120/fn/fn/fn/fn new.clj: 182
boot.new/eval1120/fn/fn/fn new.clj: 175
boot.core/run-tasks core.clj: 794
boot.core/boot/fn core.clj: 804
clojure.core/binding-conveyor-fn/fn core.clj: 1938
…
Aside from anything else, that data: {:file …}
stuff is bewildering (line 17 of what? Do I care?) and the stacktrace really isn’t useful either.
(! 504)-> cat /var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/boot.user3176105402114756413.clj
cat: /var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/boot.user3176105402114756413.clj: No such file or directory
So… not helpful
hah, yes sometimes they're created by clojure, and clojure uses temp files that are set to delete on exit
There was commentary in this ticket too https://github.com/boot-clj/boot/issues/405
In particular, there seem to be four exit cases...
Well, more to the point, there’s (at least) three types of errors: a real exception ("OMG! I haz errorz!"), a failure due to argument validation ("I can’t even run because option X is not valid"), a failure in the task due to bad input ("I started to run but failed to generate a project from the template you specified").
Only the first one really needs a stack trace shown to the end user.
Argument validation should mostly occur before the pipeline even runs and shouldn’t display a stack trace.
The last one is a grey area but I think there’s a good case for a task to make the decision.
But right now, all three have to be exceptions.
I think there definitely needs to be a way to throw an exception that Boot recognizes as "Do not print this exception’s stacktrace"
And maybe it’s just a matter of throwing (ex-info "Message" {:stacktrace false …})
and having Boot respect that?
I don’t know that we need an abort
function per se — because the normal flow of exceptions through the pipeline is fine (although some tasks really might not want certain exceptions to be caught by a task that wraps them… but that’s a separate issue).
so (when (or (nil? (:stacktrace (ex-data e))) (:stacktrace (ex-data e))) (print-stack-trace e))
?
i.e., print it if it’s not an ex-info
, or it’s an ex-info
without :stacktrace
, or when it’s explicitly :stacktrace truthy
...
It would probably be cleaner to have :no-stacktrace true
Currently boot-expectations
exits via an exception where it does want the ex-data
printed but the stacktrace is not relevant so I’d suppress it there. But boot-new
would work better with being able to print a user-level error message and just exit (with a non-zero status code)… so I think only printing ex-data
if it contains something other than :stacktrace false
/ :no-stacktrace true
/ whatever you think is appropriate… i.e., check (seq (dissoc (ex-data e) :stacktrace))
… after all, printing {:stacktrace false}
is every bit as confusing / useless as a stacktrace
Anyways, wanted to put that bug in your ear while you’re here
so some way for tasks to categorize exceptions, like you can with log levels, is perhaps a suitable approach
If the stacktrace was suppressed in the first case, it would be a much better user experience.
Mind you, even in the second case, if the concurrent exception was unwrapped(!) and we could suppress the stacktrace in ex-info
and still have the exception itself printed, that would be fine.
The nesting is… ugly and unnecessary… So we get the error message three times.
what do you think about adding a :severity
key to the ex-info, or something along those lines?
So if :severity
was present it would be print less / more of the exception / stacktrace depending on the value?
Like I say I think there’s three types of exceptions here, regardless of the verbosity level.
Although I guess I could live with two (if the ex-data
was suppressed when it was empty aside from :severity
).
i need to think about the situation where tasks want to completely terminate the pipeline
So essentially: 1. Exit with an error message (no exception information) 2. Exit with an exception (and maybe ex-data, but no stacktrace) 3. Exit with an exception and full stacktrace
If argument validation throws an exception before the task fn starts, don’t you already have that case handled?
I’d be OK with 1. and 2. above being merged into a single severity but I’d really prefer to have them separate for more control over the user experience.
this way you can have tasks that just send exceptions the way you have the general logging facility
But whose responsibility would it be to add such a task to the pipeline?
you may make a task that just logs the error or plays a sound or prints to the screen the stack trace, whatever
That’s no use if a task just returns the fs it was given instead of what the wrapped task returned tho’…
but if the metadata has been removed then that means that the notification task handled it, and boot-expectations can just continue and do nothing
so you could have a notification task that exits with error when an exception is thrown in the pipeline
@micha @seancorfield I have done a couple of things in the conversation
For example there is a PR waiting for unwrapping exceptions
Just saying, so that we reduce duplications
Also the patch for parallel tests can be useful :))
Yes agree
The metadata approach just feels like far too much subtle magic to me — especially for a task that doesn’t (currently) touch the fileset...
Boot new for example completely ignores the fileset.
but with certain guidelines about how to throw exceptions if you want to achieve more precise error behavior
As long as whatever ends up in there supports the three levels I mentioned above
OK, time to feed kitties.
(looks like it's a PR based on changes to 2.6.0-SNAPSHOT but made against master instead?) /cc @richiardiandrea
And, hey, feeding kitties is serious business here @micha !
@seancorfield it is revased on 0.2.6 so that @micha can easily merge it in
But i don't have that power
@alandipert: im aware of the musl vs glibc binary compatibility issue. i find it a bit daring to switch, but on the other hand i welcome the change big time. it's crazy how much garbage and complexity has accreted in glibc, just because it supports so many architectures which so few people or devices use... that's one of the reasons why does it take forever to recompile a whole system, which in turn leads to the need of binary distributions, etc etc. whole lot of complexity...
Can someone help pls? @danielsz for example?
I always used boot watch test
, so I'm not sure how to do it with lein
.
I see there is a lein-auto
plugin, but it's not included in danielsz/system/project.clj
, so I'm not sure how is he running his tests. Just with lein test
occasionally? (which runs of course. with a lot of errors because the way the various components are tested i guess...)
I'm asking because I'm trying to contribute an http://docs.caudate.me/adi/ component (which we use in production, but want to share with our other projects instead of copy-pasting
I settled with lein test :only system.components.adi-test
since I only had to run it a couple of times, but I would be still interested to know how are you testing it, @danielsz?
do u have the lein-auto
plugin in your system-wide lein profile?
It's kinda early in his morning according to his profile... hopefully he'll drop in and provide insight in a couple of hours?
Looking at the repo, it looks like there are all sorts of environmental dependencies for those tests... databases, network libraries, web servers... I can barely imagine what his test environment looks like
Cool got it working. Didn't realise -d and seancorfield/boot-new was a dependency (not part of core). Will have to go through the Boot documentation more thoroughly now so I can build my static website
@grounded_sage: Yeah, until it becomes part of code you’ll have to follow the instructions here: https://github.com/seancorfield/boot-new/blob/master/README.md
I’m using boot-cljs. I want to put a (println) in a macro, so that it prints at compile time (when I run the (cljs)
task in a boot repl).
@onetom: I run tests manually, exactly the same way you ended up doing. I typically run tests before uploading a new version, making sure everything works. Once a component is written, it will not often change. I'm pretty sure there's a host of test runners available for Leiningen should you need one after all. Can't be more specific cos I'm not using Leiningen much anymore.
@seancorfield: Yes, you are correct, to make all tests pass you need a machine with all those external resources installed. And I have them installed for the most part
@grounded_sage: re static website: https://github.com/hashobject/perun/
Cheers @martinklepsch I was actually planning on using Stasis because I have used it before and enjoyed it. Will see how I go.
@onetom: sorry for the late response - I think a few simple examples or just a blurb would be appropriate on the boot wiki, but the bulk of the docs should be on the project site. or maybe some generic webapp patterns with recipes from a variety of boot libs - here's how you do it with boot-http, here's how with boot-gae, etc.
@danielsz: this (alter-var-root #'prod-system component/start)
on https://github.com/danielsz/system seems incorrect because prod-system
is a system factory function, not an instance of a system:
(ns my-app.core
(:gen-class)
(:require [my-app.systems :refer [prod-system]]))
(defn -main
"Start the application"
[]
(alter-var-root #'prod-system component/start)
@micha: what's the main benefit of having boot.user
the default ns for the repl task?
i keep hitting issues regarding various symbols being already bound, like boot.core/rm
was colliding with a protocol definition of mine.
then system.boot/system
is used to define my dev
task, but then i want to access the actual running system from the repl, which is reloaded.repl/system
.
if you want to access reloaded.repl/system you could require it in the boot.user namespace
the boot.user
namespace was created because the user
namespace has special significance in clojure, and referring in vars makes it possible to use many tasks without a build.boot
@jethroksy: i had to NOT [system.boot :refer [system]]
just so I can [reloaded.repl :refer [system]]
but it makes my dev
boot task look a bit unorthogonal:
(comp
(environ :env {:http-port 3001})
(watch :verbose false)
(system.boot/system :sys #'sys/dev-system
:hot-reload true
:auto-start true)
(repl :server true))
@micha: Hmm, I'm inclined to prefer a global :target-path. What happens if I want to use it in another task defn?
@mobileink: then you can define a var:
@jethroksy: i would be curious to see how does your build.boot
look like then 😕
@mobileink: using the env as a namespace isn't a good way to do it
this is not an issue with boot either way, it's the inconvenient naming of system and reloaded.repl
@micha: ah, makes sense when you put it that way. IOW prefer genuine namespaces over the env.
@mobileink: yeah, if we have the env being like a databag of configuration options we will soon end up with all tasks tightly coupled to it, and you end up with a leiningen type project map
@jethroksy: i agree. unfortunate naming. i dont have any better idea though... and i didnt say it's boot's problem. i was just asking why is it good to start the repl in boot.user...
@mobileink: the alternative is to pass options to tasks via their constructors (when you build the pipeline) and via the fileset
@onetom: the repl task needs some default, and boot.user
is created for that purpose. Otherwise what namespace could the repl choose to start in?
@micha: glad I asked! I just finished implementing a rough version of boot-gae that uses a big :gae map in the global set-env! it works, but like a lot of the implementation I'll have to work on it a bit to make it act like a real boot lib.
You could require reloaded.repl and rename system to sys or something to prevent clashes
@micha: my question is, what do i lose - which i miss badly - if i pick some other namespace instead? i won't miss the various boot tasks because i dont need that in my dev repl since im not developing boot itself
@jethroksy: i can hack around, i know, but that would just confuse my colleagues, so it's not acceptable
;;; Copied from boot-test:
;;; This prevents a name collision WARNING between the test task and
;;; clojure.core/test, a function that nobody really uses or cares
;;; about.
(if ((loaded-libs) 'boot.user)
(ns-unmap 'boot.user 'test))
(require
'[reloaded.repl :refer [system start stop go reset]]
'[sys]
'[danielsz.boot-environ :refer [environ]]
'[system.boot]
'[zilti.boot-midje :refer [midje] :rename {midje test}])
if you make your own namespace you can set it up how you like it without needing to unmap anything
what it does for sure, is it slows down startup, if i pull in convenience libraries...
@micha: regarding boot.user, is it guaranteed that symbols defined in build.boot will always be in the boot.user namespace?
@mobileink: you can do boot -vb
in a directory that has a build.boot file, and see the generated boot.user
namespace source
@micha good question... i think im getting confused because boot-test
and midje
was requiring everything somehow
So instead of requiring that the user put :gae in set-env!, I can require that build.boot must contain a (def gae {...}) and I can always get it at boot.user/gae. Yeah?
@mobileink: I think environment variables would be nice in profile.boot
@micha: they might work. i will prep some minimal examples, but as a preview, imagine this:
src/user.clj
has (println :running-user.clj)
when i run boot watch midje
, i see:
:running-user.clj
Starting pod...
Starting file watcher (CTRL-C to quit)...
Running tests...
======================================================================
Loading (route53 utils route53-test system.components.route53 datomic.mock system.components.adi schema admin admin-test systems user)
:running-user.clj
All checks (8) succeeded.
[Completed at 01:19:54]
Elapsed time: 3.659 sec
@onetom I remember having to deal with midje requiring everything... It was one of the reasons I removed midje from that project
@nberger: can u recommend any libs which help a, verifying more complex return values? b, mocking?
@jethroksy: what goes in profile.boot? the wiki just says scripts. Clojure code?
@micha: i was making this little protocol so i can create a minimal mock route53 stuartsierra component for my very specific use case:
(ns system.components.route53
(:require [amazonica.aws.route53 :refer :all]
[com.stuartsierra.component :as component]))
(defprotocol IRoute53
(ls [this])
(ch [this action host ips]))
(defn add [this host ips] (ch this :upsert host ips))
(defn rm [this host ips] (ch this :delete host ips))
when i wanted to try it on the repl, i got:
(use 'system.components.route53)
java.lang.IllegalStateException: rm already refers to: #'boot.core/rm in namespace: boot.user
and it happened with other functions too which are only useful within build.boot
itself but not in a repl where i want to talk about my app itselfThe strategy is to use boot to do all the config stuff. So instead of maintaining web.xml and appengine-web.xml, you put the config stuff in the :gae map and boot-gae does the right thing.
@onetom sound like good discussion starters for #C08LK2DH7 :). Anyways, I like juxt/iota but perhaps you have really complex return values. And for mocking I try to survive with with-redefs
This means I need to access the config map from different tasks. As an example, the config data used to generate web.xml is also used to generate the clojure file that is aot compile to create the servlet bootstrap class files.
I could make a bunch of the config map specific to task, but I'll admit it, I'm lazy.
Plus I think that would be an additional burden on the user. Having to figure out which config data goes with which task. Maybe better to just have one config map and let boot figure out which data are used by which task.
@nberger: ah, #C08LK2DH7! sweet; i was not aware of it. i see u created it
https://github.com/juxt/iota <= wow, thx!
@mobileink: you could just (def gae {:foo "bar"})
instead of putting it in set-env!
what i liked about boot-midje is it's reloading is hyper fast. can't tell the same about boot-test, even though it warms up pods in advance... but im will revisit all these now that i got a better grasp on component and system and datomic forking.
is the lack of a "target" directory due to me being on windows or is that the norm for boot 2.6.0-SNAPSHOT ?
@jethroksy: got it, thanks. I also see ;; start global profile in my boot -vb , from ~/.boot/profile.boot. now its making sense.
@mobileink: great
@jethroksy: thanks a lot for your b.b
file! (that's what i type in intellij to find build.boot
@danielsz: which one is the recommended boot integration for environ
?
https://github.com/weavejester/environ/blob/master/boot-environ/src/environ/boot.clj
or yours?
https://github.com/danielsz/boot-environ
Use weavejester's. Mine has been merged there, and from now on weavejester's is the official one.
micha, just for the record, here is another name collision in boot.user
:
(let [{:keys [db r53] :as u} (-> (systems/mock) component/start)]
(ls r53))
java.lang.IllegalArgumentException: No implementation of method: :ls of protocol: #'boot.tmpdir/ITmpFileSet found for class: system.components.route53.MockRoute53
Hello! Is there a :injections
key in boot? a way to have simbols in clojure.core
similar to -> https://github.com/zcaudate/vinyasa#installation ?
right 😄
I wonder why I was thinking of some other magic
probably because is Sunday morning 😄
noobie question for y’all - I’m working on a boot task to port over postcss & css modules. The latter localizes css so .foo {…}
becomes .file__foo___hash {…}
and then gives you a json file with something like {“foo”: “file__foo___hash”}
I’ve got most of it working, I’m just trying to tie it all together. The question I have now is what would be the most appropriate way to retrieve that mapping in a clojurescript/clojure file? I’m thinking I would have to make a json/edn file in the boot task and add it to the classpath..? Maybe that’s a thing maybe it’s not
I’m sure I could hack something together I just want to try and get it to work ‘the clojure way ™️’
yes edn is the easiest way
so when you compile your cljs program it will have access to the edn data, in a cljs namespace
I don't know exacly the processing pipeline but if you have just a simple string to add to your processed file you can add metadata to a TmpFile and that's it
awesome, so my thinking was correct, that’s a good start. unfortunately I have no idea how to do that. Does anyone have any resources that come to mind, or examples of (simpler preferably) tasks that add files to the class path at least?
any Clojure data structure can be dumped with pr-str
to a string
then you read it with https://github.com/clojure/tools.reader
(edn/read-string (slurp "your.edn")
@bendy: you can start here: https://github.com/boot-clj/boot/wiki/Tasks#pre-tasks
oh ok, there is boot.core/add-resource
I guess you can use that
in the let binding in that example you would allocate a temporary directory in which you create your edn file
(but wait, micha knows better :D)
(deftask pre1
[...] ; task args
(let [tmp (tmp-dir!)]
(with-pre-wrap [fileset]
(let [edn (compute here)]
(spit (io/file tmp "thing.edn") (pr-str edn))
(-> fileset (add-resource tmp) commit!)))))
commit!
just does what it says...it commits you changes
ok that's what you want 😄
the main idea is that your task makes a boot-managed temp dir where it puts the files it creates
then at the end you add the directory to the fileset the previous task passed to your task
when you add a temp dir to the fileset the files in it are added relative to the temp dir root
yeah, I was thinking tmp-dir was one level lower, and then you would have tmp-dir.thing
in the classpath
later when you get that working you can do fileset diffing to cache your work and avoid recomputing things that don't need to be recomputed
oh right, presumably the output edn depends on some file that was generated in a previous task?
yeah, I’m going to have to figure out quite a bit of that, because I decided to run postcss (js) in nashorn instead of requiring a node env
and I know there’s a bunch of ways to speed that up, but that’s a problem for future me
haha well I’m sure I’ll be posting back in one of these channels when I come across that!
out of curiosity, did you have to work around any node libraries (fs, path) or async js (promises, callbacks)?