This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2015-10-28
Channels
- # aws (3)
- # beginners (14)
- # boot (222)
- # cider (9)
- # cljs-dev (14)
- # clojure (107)
- # clojure-chicago (1)
- # clojure-dev (31)
- # clojure-nl (1)
- # clojure-poland (3)
- # clojure-russia (44)
- # clojure-sg (13)
- # clojure-za (3)
- # clojurescript (81)
- # core-async (5)
- # core-logic (4)
- # cursive (3)
- # data-science (3)
- # datomic (13)
- # events (7)
- # ldnclj (6)
- # leiningen (1)
- # off-topic (4)
- # om (298)
- # re-frame (13)
- # ring-swagger (7)
- # yada (12)
@alandipert cool. Decomposing into smaller tasks sounds good. One thing about getting a failed build in ci: I want the build to fail, but I also want the junit output, so I still can't see how to get the two things at the same time
To be clear: the only way to get an exit status not= 0 in current boot is to throw? And then output to target-path is not done?
or (System/exit 1)
i can imagine a task that prints the output and then exits 1 if it's failing
there was some discussion around the best way to transmit this kind of information in a pipeline earlier this year, http://hoplon.discoursehosting.net/t/best-practices-for-creating-extensible-tasks/358, that might interest you
sorry this is turning into a bit of a goose chase
for my own purposes i'm happy to fail the build and catch upstream, but i'm open to any of these other ways, as long as running tests the simple way doesn't get any more complicated
I just built boot with boot built by boot. 🎉
so, we’re trying to fix this reload issue in boot-http
: https://github.com/pandeiro/boot-http/pull/27
basically, we’d like to get the input dirs so that reload can watch them, but the point at which we need them is outside of the context where we have a fileset
is it reasonable to use the :directories
in the boot env for this? or is there a better way to access the synchronized input dirs?
@esp1: I think (:dirs fileset)
might also be good?
i know, but we don’t have a fileset.. take a look here: https://github.com/pandeiro/boot-http/blob/master/src/pandeiro/boot_http.clj
ideally the local server would serve the same files as what would be served if you used the war task to make a warfile from your app
yes. technically tho what we’re doing is telling wrap-reload what dirs to watch in order to reload namespace changes so we don’t have to restart stuff when we’re doing development
@micha: in CI I’m downloading 2.4.0 but then it downloads 2.3.0 for some reason — any ideas why that could be? there are no properties or env vars as far as I can see: https://travis-ci.org/boot-clj/boot/builds/87880803#L94
(link updated)
that’s actually why i got involved w/this bug. i switched to boot-http and then discovered hot reloading didn’t work unless your source was in a dir path starting with “src"
and just to reiterate, (:directories boot.pod/env)
always returns the input directories? (i.e. source-dirs
+ resource-dirs
)
@martinklepsch: it downloads 2.3.0 to bootstrap itself, you need to have a boot.properties file if you want to avoid that
we could ship the loader with the release version whenever we do a new one though?
i.e. with 2.4.3 we could tell it to download 2.4.3?
Agree, it’s mostly that it’s confusing to show App Version 2.3.0 when I just downloaded 2.4.0
yeah exactly
Is there anything in the wiki about this (:directories boot.pod/env)
or is this implementation detail stuff? I'd it safe to use in libs?
@pandeiro: it's safe to use
hey @nberger i was chatting with micha about the test task and he had a really cool idea about how it could work
that might be the key to doing notification in boot generally, like we may want to start using the pattern for other notifiers also
@martinklepsch: how do we want to handle the boot.sh for homebrew?
i guess we can make tags for it in github, which will allow us to upload binaries to gh releases
@micha: Why not just do it like before?
because it's in one source repo with the boot.core lib and what boot.core version gets bumped the boot.sh might not necessarily be changed, so it's confusing to label the "exact same" binary with a different version? (or something like that?)
also @martinklepsch your boot-builds-boot build.boot
file is incredible
it doesn't know anything about boot other than that there is a boot.App class with a main() method
(i havent realized until now that the loader is another component in the game and it's different from the boot app itself... but then again, im not fluent at all in java to understand such internals. yet.)
@alandipert: I'm interested, how would that work?
@onetom: basically the only reason to change the loader is to fix a bug, never to add functionality
@nberger: the thing to consider is the fileset returned by the next task
imagine the test task adds a test report to the fileset, and passes it to the next task. now, imagine it also retains the fileset returned by this call. then it looks inside the returned fileset - if the report is there, it does something with it, like throw or print
@micha: what do you think about a separate repo for the loader then different versioning schemes would clearer separated
@nberger: now, downstream tasks can choose what ot do with the report, and whether or not to "bubble it back" to the task that added it
@micha: thanks for the praise, I’m happy about any suggestions (style, idiomatic, …)
@alandipert: I think I'm following you, but I'm not sure how it helps to solve the clojure.test + junit scenario. One important thing to note on that scenario is that the tasks needs to fail (not (zero? exit-status))
but also target needs to be synced, because we want the junit reports to be in the fileset but at the same time we are especially interested in getting the reports when the build fails
@micha: I’m working on a change that checks if the current commit already has the given tag and does not try to create it if so
@micha: does that make sense to you?
@alandipert: but yes, I like the two-stages task idea, it helps to make the code more clear and extensible
@micha: maybe we wait with the split until after we merge boot building?
It's still that I can't see how to fix the problem of putting a report into the fileset and expect it to be materialized into target-path
, even if the pipeline fails. I was thinking of throwing a specific exception that would be recognized by boot and in that case the target would be synced
@nberger: ah, i see what you mean now - we need the test report no matter what
boot test; [[ -e "target/failures.xml" ]] && exit 1 maybe?
i'm not sure we can support both failing and not failing
and would need t orely on an external check of failed-ness in this case
Checking the existence of failures.xml might be brittle I think... and it makes so that we have to be careful to not generate the report when there are no failures, which is not always desired
another unsavory option that comes to mind that @micha will hate is osme kind of after-sync hook... where we could exit1
Both fail but no fail... I see the issue there... I was thinking of relaxing the rule of "when we fail, we don't sync the target". It would be opt-in of course, the default would always be to not sync it
it's kind of like we want a "finally" clause for boot itself that syncs the fs
yes... one pesky detail is that the task that throws must send the resulting fileset as part of the exception?
although... soon, there will be an explicit target task
which would sync before returning the fileset
or at least, it could
I'll keep thinking on this task. In the meantime I'll use the simplified version that outputs right to the filesystem instead of the fileset. Let's see if that target task helps, or we'll arrive to a solution one way or another
For anyone interested: https://github.com/hashrocket/boot_devcards_example
Credit due to @juhoteperi for the saapas example. I started with that build.boot and modified as necessary.
@jgdavey: very nice!
Have you guys ever seen this:
clojure.lang.ExceptionInfo: java.lang.NoClassDefFoundError: org/apache/maven/wagon/Wagon, compiling:(NO_SOURCE_PATH:0:0)
data: {:file "/tmp/boot.user6112313111468063070.clj", :line 15}
clojure.lang.Compiler$CompilerException: java.lang.NoClassDefFoundError: org/apache/maven/wagon/Wagon, compiling:(NO_SOURCE_PATH:0:0)
java.lang.NoClassDefFoundError: org/apache/maven/wagon/Wagon
java.lang.ClassNotFoundException: org.apache.maven.wagon.Wagon
...
clojure.core/eval core.clj: 3081
boot.aether/load-wagon-mappings/fn/fn aether.clj: 193
only happens on build machine of course. Not on my machine, not on the sysops' team machines, etc. Only on one machine.
@nberger: sounds good. thanks again for helping think it through
@alandipert: thank you for the help
Is there a way to get boot do dump the entire classpath it is providing to any given pod?
how can I get a reference to this class? https://github.com/eclipse/jgit/blob/9e5380e7fb797e83b6f6b5754249bea4cae534e7/org.eclipse.jgit/src/org/eclipse/jgit/lib/ObjectIdRef.java#L106
@chrisn: running 1.8. Don’t remember seeing your issue. Do you always get this?
what’s boot -V? or does that also not work?
ubuntu@ip-10-25-167-117:~/workspace/pi-build-catalog$ boot -V docker-target
#
#Wed Oct 28 16:14:38 GMT 2015
BOOT_CLOJURE_NAME=org.clojure/clojure
BOOT_VERSION=2.4.2
BOOT_CLOJURE_VERSION=1.7.0
#App version: 2.4.2
@chrisn: tried running built in tasks I assume?
hrm. you could try rm ~/.boot/cache
but I don’t see how that would change something
The jvm isn't lying, if you set the boot_jvm_options to be:
-verbose:class
then grep -i for wagon you only get aether stuff.So for some reason on this exact machine the classpath isn't getting setup correctly on the pod.
While it is getting setup correctly on every other machine. Also, lein deploy will work on this machine for this project meaning it is using our private s3 repo from leiningen.
I’m setting up a project that mainly consists in generating clojure source files from json files. I have a direct mapping of json-file -> processing-pipeline -> output clj. I’d like to define a task for this whole process, but I’m not sure if I really need to work with filesets for this, since I have this pre-determined wiring of inputs to outputs?
@jfntn: sounds like a perfect case for writing a function that's fileset in, fileset out - a task
BAM build passes!
@alandipert: so I’d just write the files to a hard-wired path in my src dir, and then commit! the fileset?
more like, add the directory containing your json as a source-path, and then run your task that does the transformation
@jfntn: oh wait... you want to feed the files back into your project code?
jfntn: do you want them on the classpath or what is the ultimate thing you want to do with these clojure files?
I could do without adding them to the class path, ultimately there will be another task that will jar up the generated code
perfect boot scenario! 👍
@jfntn: basically you write those generated clojure files to some tmp-dir!
(see above), add this dir as resource via add-resource
and then call the jar
task afterwards
@jfntn: out of curiosity - what kind of project is this? json -> clojure sounds interesting 😄
A bunch of HTTP helpers generated from https://github.com/for-GET/know-your-http-well and some RFCs too if I can some clean xml files 😛
@jfntn: https://github.com/boot-clj/boot/wiki/Filesets#examples sounds very close to what you want, just need to bring your own compile-lc!
ok great, that should be enough to keep things moving, thanks @alandipert, @martinklepsch !
cool thought about designing stuff: http://cl-su-ai.lisp.se/msg05941.html
haha @alandipert were you born yet when that email was written? btw is CL finished yet?
no & yes, but they didn't listen to stallman
http://cl-su-ai.lisp.se/msg00116.html also, wish clj did more of this
Still working on the classloader issue I have on one random machine. What I would like to do is log clearly every time add-classpath is called in pod.clj. I guess other relevant information would be which classloader it is getting from the current thread or some identifying information about the classloader stack. Is there some conceivable reason that either the correct dependency information for the s3 plugin isn't getting added to the pod or the classloader is ignoring the add url command?
Or any other scenario we can think of where the wagon dependency magically isn't getting found?
For reference, when it works we get:
[Loaded org.apache.maven.wagon.Wagon from file:/home/chrisn/.m2/repository/org/apache/maven/wagon/wagon-provider-api/2.2/wagon-provider-api-2.2.jar]
@chrisn: i wonder if the maven repo on the build machine is corrupt somehow
@chrisn: you could test by setting BOOT_LOCAL_REPO to something different
also that
very cool discovering :(directories boot.pod/env)
thanks @esp1 @alandipert
i'm amazed i never looked at that before but it's a testament to how well boot manages this that i never had to
I’m excited! I want to merge this now! Who wants to test?
sure sure
Workflow with —watch
works pretty well mostly. Sometimes I get things like java.lang.Exception: can't resolve symbol (boot.jar/spit-jar!)
during the first uberjar build, don’t know what’s going on there.
when invoking boot tasks with edn
opts, it's necessary to do :some-edn-opt (pr-str {:a true :b [1 2 3]})
even inside e.g. build.boot?
it's illegal to do (apply some-task {:some "a" :args "b"})
? is there another way i can call a task with an indefinite map of opts?
👍 good question, wanted to do that too the other day
@pandeiro: try (apply some-task (mapcat identity {...}))
the problem is the map seqs into a seq of pairs
oh that makes sense!
(apply some-task {:x y}) == (apply some-task '([:x y]))