Fork me on GitHub
#boot
<
2016-09-14
>
seancorfield00:09:04

OK, the first problem is your boot.new.readux_starter namespace — should be readux-starter (`_ in filename, -` in namespace).

seancorfield00:09:29

That gets you to this exception

Generating a new project called pseudapp2 based on the 'readux-starter' template.
Template resource 'boot/new/readux_starter/README.md' not found.

seancorfield00:09:01

And that’s because you have - in the folder name instead of _

seancorfield00:09:59

Once both of those things are fixed, it generates correctly.

seancorfield00:09:23

https://github.com/seancorfield/boot-new/issues/26 <— I’ll make error handling better around that

michael.heuberger00:09:06

does anyone know a trick how i can publish a local package to my local .m2 repository and fetch it within another boot project?

michael.heuberger00:09:17

this to double check if it works fine before publishing it to the wild

seancorfield00:09:45

@pseud with the latest boot-new (0.4.7) you’ll now get this

boot -d seancorfield/boot-new new -t readux-starter -n pseudapp2 -V 0.1.2-SNAPSHOT
Retrieving boot-new-0.4.7.pom from 
Retrieving boot-new-0.4.7.jar from  (44k)
Retrieving maven-metadata.xml from  (1k)
Retrieving boot-template-0.1.2-20160913.205445-1.pom from 
Retrieving boot-template-0.1.2-20160913.205445-1.jar from  (8k)
Found template readux-starter but could not resolve boot.new.readux-starter/readux-starter within it.
clojure.lang.ExceptionInfo: 1
    data: {:file
           "/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/boot.user7778462556735229335.clj",
           :line 17}
             boot.App$Exit: 1
boot.new-helpers/resolve-template/invokeStatic  new_helpers.clj:   78
which gives you the specific symbol name it is trying to resolve.

seancorfield00:09:15

@michael.heuberger a task like this should work

(deftask build []
  (comp (pom) (jar) (install)))

seancorfield00:09:50

Then boot build will make a JAR and put it in your local ~/.m2/repository per the task-options! you’ve set for pom and jar.

seancorfield00:09:09

(that was how I debugged and resolved @pseud’s problem above btw)

michael.heuberger00:09:13

@seancorfield thanks, i did that. but then, the other project can’t find it when i declare it as a dependency there

michael.heuberger00:09:34

how do you declare a local dependency in build.boot?

michael.heuberger00:09:59

so that it will look up in .m2/repository and not on i.E. clojars etc

seancorfield00:09:39

It just does it automatically.

seancorfield00:09:33

It’s just a regular dependency. Because you have it cached locally, boot (or rather aether) won’t look on Clojars etc.

seancorfield00:09:02

The only caveat is snapshots since it will check each day for a new version (I believe).

michael.heuberger00:09:45

@seancorfield well, it doesnt. and i see this red error clojure.lang.ExceptionInfo: Could not find artifact …

michael.heuberger00:09:58

it always looks up in clojars, not locally

seancorfield00:09:49

are you sure you have the correct options for pom and jar when you install it locally?

michael.heuberger00:09:14

i can find it fine in .m2/repository ...

michael.heuberger00:09:21

it’s just the other project that can’t find it

seancorfield00:09:49

(I do this all the time, BTW, and it works just fine — I did it just 30 mins ago to test and debug the readux-starter template)

seancorfield00:09:07

I’d have to see your build.boot file for the project you’re building & installing locally and the dependency in the other project’s build.boot file — and perhaps details of the tree / files in ~/.m2/repository to be able to debug this any further for you.

michael.heuberger00:09:20

probably there are other external factors that can mess with this

pseud05:09:22

@seancorfield Thanks a lot for looking at this - hopefully it’s not terrible form to leave a question overnight as I did 🙂 I thought some of it was due to the dash/underscore debacle but it didn’t seem to resolve things for me yesterday. I’ll look into your instructions properly once I’m off work. It’s much appreciated 🙂

seancorfield05:09:44

NP. The error message wasn't great... It should be better now 🙂

pseud05:09:37

@michael.heuberger wrt to finding local jars. boot-new looks in your local repo’s first, then at clojars, then a whole list of places. I found that out by 1. bumping the version of my library to something more than what was available at clojars 2. trying to fetch it in another project and failing 3. installing it locally (`(comp (pom) (jar) (install))`) 4. trying to fetch the project again and succeeding. Now, keep in mind that boot will ignore your bumped version if it’s a snapshot. In case of boot-new I could circumvent that by passsing -S (look for a snapshot) or explicitly specifying the desired version with -V

seancorfield05:09:19

☝️:skin-tone-2: Thank you for that @pseud

pseud05:09:35

You are very welcome 🙂

onetom08:09:30

@martinklepsch is there an example project for https://github.com/martinklepsch/s3-beam? i would like to have a feel for it 1st and inspect the various bits of data passing thru it, then i would probably make javelin/castra version of it

martinklepsch09:09:46

@onetom no example except for the stuff in the readme

mitchelkuijpers12:09:33

java.io.FileNotFoundException: Could not locate boot/core__init.class or boot/core.clj on classpath., compiling:(cloudformation/boot_cloudformation.clj:1:1)
I get this error when I try to run boot-test any idea what causes this? To me it looks like it tries to load boot.core in the test pod and that fails…

dominicm13:09:14

@mitchelkuijpers That looks like boot-cloudformation (what is that?) is to blame? I'd guess that boot-test is loading up that file to check if there are tests in it or something like that

mitchelkuijpers13:09:31

Yeah that seems to be the problem

mitchelkuijpers13:09:52

boot-cloudformation are some of our internal tasks we use to run cloudformation commands

mitchelkuijpers13:09:05

Now i am trying to exclude it with sift

mitchelkuijpers13:09:58

sifting it does not seem to work hmm

alandipert13:09:03

@mitchelkuijpers boot.core isn't available by default inside pods

mitchelkuijpers13:09:31

@alandipert Yeah i have just added it as a dependency, took a while to realize that

alandipert13:09:33

(and test task runs tests inside pod)

alandipert13:09:55

np. be also advised that depending on what you're doing with boot.core in a pod, it might not work

alandipert13:09:08

but if it's fixed then you don't need to worry about it 😄

mitchelkuijpers13:09:37

We are not really doing anything with these tasks in our tests. boot-test is just very commited to compiling the thing 😛

dominicm13:09:38

@alandipert what's the best way to ignore a namespace when using boot-test, out of interest?

alandipert13:09:36

it takes a :namespaces key for the namespaces to run, is that what you mean?

alandipert13:09:49

oh, and also :exclusions to do the inverse

dominicm13:09:28

@alandipert So instead of :exclusions, it would work to do :exclusions '[boot-cloudformation] (or however the argument looks to boot-test)?

alandipert13:09:01

i think so. i tend personally to explicitly give it my test namespaces to load

alandipert13:09:09

i.e. using :namespaces instead

alandipert13:09:38

there's also the :filters option which works like (test :filters '[(re-matches #".*test.*" (name %))])

alandipert13:09:01

which in retrospect is too weird to have included

dominicm13:09:01

That makes sense I think. A way to test all test namespaces

alandipert13:09:53

yeah - all the ones with test in the name

zilti15:09:09

How exactly does Boot mess with the classpath upon modifying and committing the fileset? I have a piece of code which I want to access both static files and a file generated by a boot task before running the main program. My problem: when I do (io/resource "data") at compile time, I can access the static files, but not the generated one. When I do (io/resource "data") at run time on each call, I can only access the generated file, but not the static ones.

alandipert15:09:48

@zilti sounds like your static files are in :source-paths? that means they don't have "output role" and woulndn't make it into a jar

zilti15:09:26

@alandipert well, the folder they're in is in both :source-paths and :resource-paths. So, that's the problem?

alandipert15:09:33

could be, try putting everything in resource-paths

alandipert15:09:47

source-paths are for files that are to be "consumed" during the build, like .java

alandipert15:09:57

or .clj if you're AOT-ing

zilti15:09:18

@alandipert Hmm, I've put everything in :resource-paths only now, still the same problem...

zilti15:09:51

Does the task order matter at this point? I basically have (comp (run-application) (watch) (gen-file)) since I want the task to re-generate the file upon change.

alandipert16:09:46

oh hm - so at compile time, you're not accessing the files using the fileset api?

alandipert16:09:20

also what is the task that does the transform?

zilti16:09:55

I access the files after the transform happened for the first time, using the normal http://clojure.java.io API, and the transform simply evaluates a clj file and stores the result into a file in the fileset (first in a tmp-dir, then committing it, exactly the way it is described in the fileset tutorial)

alandipert16:09:08

what is the differenc between compile time and run time in your case? boot entrypoint vs. java -jar or servlet?

zilti16:09:17

No, I always run the application via boot. The difference is that in the "compile-time scenario" the (io/resource "data") is in a "let" expression around the function, and in the "run time scenario" it is inside that same function.

alandipert16:09:03

the output of yuor task needs to be on the classpath to be useful? or if you had a File object, would it be useful?

zilti16:09:46

I don't want to push File objects around the whole application whenever the input changes, so, I'd prefer not to

alandipert16:09:05

so i think the underlying complication here is that your compile and run times share a classpath

alandipert16:09:35

i would tackle this by running the app in a pod

alandipert16:09:25

so working back from the pod, a task is responsible for creating/starting the pod with the :directories argument, which is a list of dirs you can tell pods to put on their classpath

alandipert16:09:52

i would use the target task to sync the FS to a temp dir, and then use that dir as the :directories for the app pod

alandipert16:09:23

at this point, your app sees only and exactly what's in the fileset at the end of your build process

alandipert16:09:51

this would also allow you to have different sets of dependencies for build vs run

alandipert16:09:10

i haven't done anything exactly like this so its just my intuition, there might be blockers i can't think of

alandipert16:09:42

circling back to your original problem, i think the issue is that all the files you need at runtime are in the fileset, but they're not necessarily on the classpath

alandipert16:09:56

i use that to dev java in a watch/reload fashion

micha16:09:44

@zilti did you say that :source-paths and :resource-paths share a directory?

micha16:09:02

if there is any overlap between them then undefined behavior will result

micha16:09:07

there should be an exception thrown

micha16:09:08

eg. :source-paths #{"src"} :resource-paths #{"src/foo"} is a no-go

zilti17:09:01

@micha I removed it, but it didn't make a difference 🙂

zilti17:09:16

@alandipert Thanks, I will look at that!

zilti17:09:03

I solved it by adding "(add-resource (io/file "resources"))" now. 🙂

micha17:09:57

@zilti is "resources" not in your :resource-paths?

zilti17:09:21

@micha yes, it is. That's why I am so confused about all this in the first place...

micha17:09:35

yes that isn't going to work, something is wrong

zilti17:09:54

Well, it does work now, so there's that

micha17:09:00

usually you use add-resource to add temp dirs

micha17:09:29

well it may work at the moment, but it's definitely going to confuse things

micha17:09:49

do you have tasks that are writing to "resources" dir?

zilti17:09:51

@micha: And that's exactly what I did. I added the path "resources" to the fileset

zilti17:09:04

yes, I do have one writing there

micha17:09:08

ah that's the problem

zilti17:09:18

Or, not directly there, but into the fileset's equivalent location

dadair17:09:34

Anyone have any references/examples of how to convert a lein template that uses profile-hints to boot? eg lein new duct [app_name] +hint1 +hint2 +hint3? I’d like to submit a PR to duct for using boot but I haven’t seen examples of this functionality.

micha17:09:01

what are profile hints?

micha17:09:07

@zilti so you want to undo what that task did?

micha17:09:20

like the task overwrites something in resources, and you want to undo that?

zilti17:09:04

@micha The way I do it now is: 1. add "resources" to the fileset 2. launch the application 3. the task writing a file into "resources" (not overwriting) is run upon changes

dadair17:09:13

@micha project templates with conditional code… eg. {{#cljs?}}[org.clojure/clojurescript "1.8.51"]{{/cljs?}} see https://github.com/duct-framework/duct/blob/master/lein-template/resources/leiningen/new/duct/base/project.clj

micha17:09:17

@zilti i work this way all the time wihtout needing to add anything via add-resources

micha17:09:35

like for example starting a pipeline with the cljs compiler and a jetty local dev server

micha17:09:46

you can try this yourself

micha17:09:57

the cljs task will write new files to the fileset

micha17:09:18

like boot watch cljs serve for instance

micha17:09:35

when you change a source file it'll recompile, which adds files to the fileset

micha17:09:40

then the server can see them

micha17:09:43

on the classpath

micha17:09:04

how is that different from what you're doing?

zilti17:09:43

@micha Hmm, my ordering is different: I basically have "boot add-resources-and-launch-application watch writing-task

micha17:09:59

you should put your application at the end

micha17:09:36

then it will not start until the things it needs have been created

micha17:09:46

but you want to start it in a background thread of course

micha17:09:51

so it doesn't block the pipeline

micha17:09:09

and in a delay so it doesn't start every time the watch fires

micha17:09:54

`
(deftask doit
  []
  (let [start (delay (future (start my server)))
        stop  #(stop my server)]
    (cleanup (stop))
    (with-pass-thru fileset @start)))

micha17:09:59

the delay makes sure the server is only started once

micha17:09:17

and the cleanup part makes sure it's stopped

micha17:09:27

(when the pipeline ends)

dadair18:09:52

@micha please disregard my previous query, I was not thinking about this in the right way. I can just generate the boot file!

micha18:09:22

@dadair whew! excellent!

micha18:09:57

i saw reader literals and started backing away slowly 🙂

micha18:09:08

lol joking of course

zilti18:09:11

@micha hmm your solution brings with it that I again don't get the static files ^^

micha18:09:31

@zilti static files?

zilti18:09:47

@micha the files that already are in the folder and aren't generated by the task

micha18:09:32

so if you add show -f right before your server task in the pipeline, do you see those files?

micha18:09:25

perhaps you have a task that is removing those files frrom the fileset

micha18:09:02

or they're not in the fileset in the first place (eg. not in :resource-paths)

zilti18:09:02

Yes, with "show -f" I see them.

micha18:09:28

so you can see them with show -f but your task doesn't see them on the classpath, is that the situation?

micha18:09:52

and the files are in a directory that is in :resource-paths?

micha18:09:31

if you can see them with show -f then they're in the fileset

micha18:09:51

but if you have them in for instance :asset-paths they won't be on the class path

zilti18:09:42

Yes and yes. Actually, the application does see the static files, but not the dynamic ones, when I do (io/resource "data") (data is a directory inside "resources") as a top-level in my .clj, but the inverse is true - sees the dynamic file, but not the static ones - when I do (io/resource "data") inside a function.

micha18:09:04

there are no directories in the fileset

micha18:09:05

only files

zilti18:09:10

Except when I do a (add resource (io/file "resources") inside my build.boot

zilti18:09:26

Then, everything works fine.

micha18:09:26

there is no resource associated with a folder in the fileset, or the classpath, so i'm not sure what (io/resource "data") would mean

micha18:09:55

the fileset is a set of files with associated paths, basically a key value store

zilti18:09:21

Yes. But inside my application I access them with http://clojure.java.io/resource.

micha18:09:07

maybe you are not calling commit! somewhere

micha18:09:19

so the fileset doesn't actually reflect the actual classpath

micha18:09:25

or vice versa

zilti18:09:18

Oh I do call commit! whereever it is necessary 🙂

micha18:09:07

i don't understand what is different about what you're doing from what a typical cljs build does

micha18:09:32

it's very common to be creating new files in the fileset alongside existing files

micha18:09:50

and i have never seen files become invisible from the classpath before

micha18:09:29

i mean if the files are in the internal boot directories then they will be on the classpath

kenny19:09:31

Is there an ETA on 2.7.0?

arohner22:09:54

noob question: why does target/ contain tons of source files? Should it?

micha22:09:33

@arohner it will contain any files in the fileset that have the "output" role

micha22:09:51

that is things from :resource-paths or added via add-resource to the fileset

micha22:09:15

if you don't want to have certain files in there you can use the sift task to filter them out

arohner22:09:29

it looks like it also contains unzipped dependency sources files

micha22:09:11

is this an uber jar?

micha22:09:24

it must be i guess if it contains unzipped dependencies

micha22:09:57

and yes, that's supposed to be like that, as the target dir will contain all the files in the fileset with the "output" role, which is the same as the files that would be included in a jar if you were to make one of those

arohner22:09:25

yeah, it is an uberjar

micha22:09:46

yeah when you use the uber task you unzip all those dependencies into the fileset

micha22:09:02

so unless you remove them they'll be in there when you call the target task

micha22:09:06

and it will write them out

micha22:09:44

tasks are not coupled, so the target task just does one thing and one thing only: it writes out files that are in the fileset to some directory

micha22:09:54

it knows nothing about uberjars or anything else

micha22:09:13

likewise, the jar task doesn't know anything about uberjars either

micha22:09:27

you can make an uberjar by chaining the uber task before the jar task

micha22:09:48

the jar task simply creates a jar file that contains the files in the fileset

micha22:09:13

and the uber task only knows about unzipping dependencies and adding them to the fileset

micha22:09:25

so to make an uberjar you compose uber and jar tasks

micha22:09:57

if you want a pom.xml in your jar for whatever reason you compose with the pom task, that only knows about creating pom.xml files and adding them to the fileset

micha22:09:24

and at the end, the target task only knows to take all the files in the fileset and write them to a directory

micha22:09:37

so if you do boot uber target then it will write out all the files

anmonteiro23:09:12

having problems with data_readers

anmonteiro23:09:24

doesn’t Boot merge several data_readers.clj?

anmonteiro23:09:39

it’s either not merging them or not loading any at all

anmonteiro23:09:52

even after adding (load-data-readers!) to my build.boot

micha23:09:53

try (boot.pod/resources "data_readers.clj")

anmonteiro23:09:22

@micha turned out to be PEBCAK

anmonteiro23:09:43

moved (load-data-readers!) before requires

richiardiandrea23:09:19

lol what's a PEBCAK ah ah sounds some funny internet acronym 😄

anmonteiro23:09:37

@richiardiandrea "Problem Exists Between Chair And Keyboard"

richiardiandrea23:09:24

always nice to learn new cool ways of blaming yourself 😄