Fork me on GitHub
#boot
<
2016-06-23
>
kenny03:06:54

Is is possible to include a license/docstring above your compiled cljs code using the cljs task?

sekao04:06:29

i’m trying to use boot as a library by calling boot.App/main. i’m currently successfully calling boot tasks this way, but it will sometimes randomly fail with an opaque error in boot/main.clj: Pop without matching push. i would love any help; i understand what i’m doing is unconventional but it seems like it should be achievable.

martinklepsch07:06:58

@kenny: You could try the preamble compiler option

kenny07:06:14

@martinklepsch: You that did the trick, thanks!

pesterhazy13:06:31

In a combined clj+cljs boot file, how do I get the repl prompt (`:server false`) to show up? I don't see a prompt with what I have:

(deftask dev-repl []
  (comp (development)
        (repl)
        (watch)
        ;; (cljs-repl)
        (reload)
        (build)
        (run-server)))

pesterhazy13:06:59

To be clear, I want the clojure repl to show up (`repl`) in the terminal that runs boot dev-repl

pesterhazy13:06:28

@martinklepsch: so call (repl :server true) and later (repl :client true) in the same boot pipeline?

martinklepsch13:06:12

do you have any task-options! calls for the repl task?

martinklepsch13:06:24

just invoking (repl) should bring up a prompt

pesterhazy13:06:05

that's what it did, until I added the cljs-related tasks (brazenly stolen from tenzing)

pesterhazy13:06:49

hm. so boot development repl works, boot development repl wait doesn't (it doesn't give me a prompt)

martinklepsch13:06:35

right the wait task will block forever not "finishing" the pipeline

pesterhazy13:06:44

same for boot development repl watch build

pesterhazy13:06:15

@martinklepsch: but shouldn't repl be executed before that?

martinklepsch13:06:51

I'd expect the server to start but the prompt probably attaches after the pipeline

pesterhazy13:06:57

ah so it starts the server first, then executes everything, and then attaches the prompt to the nrepl

martinklepsch13:06:25

I think so, yeah

martinklepsch13:06:02

I not sure how watch and repl interact you might want to try different orders of these

pesterhazy13:06:12

boot watch repl, unfortunately, blocks and doesn't execute the rest of the pipeline

pesterhazy14:06:25

as a workaround I'll use boot repl in one window and boot repl -c in another, though that's not optimal of course

martinklepsch14:06:34

@pesterhazy: fwiw, I use inf-clojure and in this case I use boot repl -c as "jack-in" command making the workflow pretty much cider-like (just without cider)

pesterhazy14:06:34

@martinklepsch: so you're opening two terminal windows?

pesterhazy14:06:13

by the way I wish there was a boot repl -c-in-a-jar which starts up faster than boot repl -c

pesterhazy14:06:12

perhaps written in planck for turbo speed?

pesterhazy14:06:25

or in ocaml or something 🙂

martinklepsch14:06:46

@pesterhazy: I open a terminal window and the other "terminal-window" is opened in emacs

pesterhazy14:06:30

yeah I still haven't gotten used to using terminals from inside emacs

martinklepsch14:06:42

@pesterhazy: well it turns into a repl immediately which you can send forms to and all that

martinklepsch14:06:23

the terminal window reference is more originating from the fact that this essentially just starts the boot repl -c process with which it communicates via stdin/out

richiardiandrea15:06:53

Btw the way @pesterhazy, my lambone template does exactly that, two repls in one terminal where you can connect to. I wanted to completely separate the backed classpath from the frontend classpath, following juxt's edge. Currently you don't strictly need two repls if you have every dep (frontend and backend) on the classpath (like lein on standard boot), but I prefer this way...in any case you can check the dev-backend task for how to write a non-blocking Repl task

dominicm15:06:09

Is boot 2.6 officially out?

martinklepsch15:06:40

@dominicm: it's tagged on github so I guess yes

richiardiandrea15:06:06

Yes it is been using it for a while

dominicm15:06:52

Only a month late shrugs

richiardiandrea15:06:19

Yeah a while is a month wow it feels longer lol

dominicm15:06:34

Looks like boot-reload 0.4.0 doesn't work with it, that's the only reason it's even come up.

dominicm15:06:12

Time to pin the boot version!

micha16:06:32

@dominicm: is it the template issue?

micha16:06:50

you can use ns-unmap to workaround that

richiardiandrea16:06:37

@micha a question about https://github.com/adzerk-oss/boot-test/issues/26 ...is it possible that it is because there is some missing cleanup to add?

micha16:06:56

@richiardiandrea: have you tried putting visualvm or something like that on it to see what's using the memory?

richiardiandrea16:06:12

it is using the pod pool, I'll try that

micha16:06:45

i can't think of a way that the pod pool could be causing this

micha16:06:05

unless something is holding references to pods or something

micha16:06:20

so they can't be eligible for GC

richiardiandrea16:06:29

kk I am going to inspect the code a bit better

micha16:06:57

a tool like visualvm will show you what's holding the objects

micha17:06:13

you'll be able to watch it grow till you get the error

richiardiandrea17:06:25

yep...checking...I'll report back

richiardiandrea18:06:15

so basically the therad clojure-agent-send-off-pool-1 is doing something funny and growing and growing. Dumping the threads reports that it is the watch task

richiardiandrea18:06:14

triggering manual gc does not shrink the memory usage of the thread

micha18:06:06

strange because we're not using agents in there

micha18:06:16

oh the atom

richiardiandrea18:06:36

maybe the boot.watcher stuff? Reading...reading

richiardiandrea18:06:58

yeah by dumping the thread I see that is running my tests

dominicm19:06:49

@micha: It's better to just update boot-reload really 😛

micha19:06:05

haha agreed

lwhorton19:06:29

if I am publishing a library foo, and I list :source-paths… no other project that lists :dependencies ‘[foo] can find the namespaces on the classpath. However, if I list the foo code under :resource-paths, everything works dandy. this makes me realize I have no idea what the difference is between these two keys. can I look that up somewhere in more detail?

micha19:06:29

basically there are two "roles" a file in the fileset may be assigned

micha19:06:46

so that means 4 permutations of those roles (they're not mutually exclusive)

micha19:06:19

that link i sent has a table of which *-paths thing corresponds to which permutation of the two roles

lwhorton19:06:07

so if I understand this correctly, by the nature of this being a library for others to consume, i need to setup the code as :resource because the code needs to be emitted, used by a packaging task, and show up as a final artifact

lwhorton19:06:27

and source files (input) are not emitted as output because ..?

micha19:06:29

right, and be on the classpath when you build, presumably

micha19:06:45

because :source-paths do not get the "output" role

lwhorton19:06:50

they’re expected to be consumed and spit out to something else?

micha19:06:52

they only have the "input" role

micha19:06:07

:resource-paths have both "input" and "output"

micha19:06:40

yeah things with "input" role will be on the classpath

micha19:06:46

so they can be consumed

micha19:06:57

but the packaging type tasks will not package them up

micha19:06:44

in retrospect i think it's maybe more complicated than it needs to be

micha19:06:57

because the sift task could really handle removing things

lwhorton19:06:35

hmm, ill have to think on it some more to make sure I really can digest it

micha19:06:30

we identified two characteristics of files you work with in a build

micha19:06:36

input files, and output files

micha19:06:46

input files are ones you consume during the build

micha19:06:51

or to run a program, etc

micha19:06:56

things you read

micha19:06:27

output files are artifacts that are produced that will be delivered to some other process

micha19:06:31

after the build is complete

micha19:06:53

but a given file can be both, or neither, too

lwhorton19:06:57

normally I work in cljs land, so to me ‘input’ is a cljs file, and output is a js file— and that makes sense. the thing i’m consuming is javascript. what’s more confusing to me is when publishing a boot-task, the thing i’m outputting is just more clojure code to be consumed elsewhere. i guess according to boot’s definition that has to be :resource-paths to mark as ‘output’.

lwhorton19:06:13

so the difference being that a boot-cljs task probably marks the produced js as ‘output’ role, right?

micha19:06:39

that's corrrect, it adds them to the fileset with add-resource

micha19:06:18

so they also have the input role too, so you can further process the js with another task if you want to

fenton19:06:51

trying to use boot-cp, but it is throwing a 'conflicting dependencies' warning, how to force it to create a proper dependencies classpath file?

richiardiandrea19:06:19

@fenton: hey! I think that task has been designed to throw/report conflict, it should write down a file with -w -f your-file.

fenton19:06:02

@richiardiandrea: % boot with-cp -w -f deps.out doesnt' write a file when there is a warning. hmmm...

micha19:06:46

you can use the boot show -p task to see which dependencies are conflicting

micha19:06:09

then add dependencies to override the conflicting ones

micha19:06:16

or use :exclusions

fenton19:06:16

@micha: how to resolve deps, explicitly stating them in build.boot doesn't seem to help?

micha19:06:29

can you paste what you're doing so far?

richiardiandrea19:06:30

yeah you need some chiseling 😄

fenton19:06:36

k gimme a sec to come up with a simplified example.

fenton20:06:32

ah, ok, got it figured out with exclusions...

micha20:06:08

boot show -p and boot show -d should help with that

micha20:06:34

to visualize the dep graph

fenton20:06:46

lein does: lein classpath without having to do all the exclusions stuff manually...thats handy.

micha20:06:54

@fenton: you can omit the --safe/:safe true option

micha20:06:00

can you paste what you have?

micha20:06:24

it should only throw the exception when you tell it you want that

fenton20:06:28

boot with-cp -w -f deps.out

micha20:06:46

you must have some other code that configures it

micha20:06:05

like profile.boot or build.boot

richiardiandrea20:06:15

(def exclusions ....)

(task-options! pom {:project 'acu
                    :version +version+
                    :url ""
                    :description "FIXME: write description"
                    :license {}}
               with-cp {:file "cp"
                        :safe true
                        :dependencies frontend-deps
                        :exclusions exclusions})
My conf, not that bad 😉 I lauch it with boot with-cp -w

fenton20:06:27

might be old version lemme check

martinklepsch20:06:41

@fenton: lein classpath isn't intended to throw if there are conflicts and just resolves them similar to how boot show -c/-C work (I see with-cp isn't either ...)

fenton20:06:04

yup i had an old version...srry

micha20:06:34

@fenton try this: boot -BP -d ring -d hoplon show -p

micha20:06:51

that will show conflicts

micha20:06:04

then do this: boot -BP -d org.clojars.micha/boot-cp -d ring -d hoplon with-cp -w -f foop

micha20:06:16

that wil write the file, conflicts and all, without throwing an exception

fenton20:06:26

ahh, that is neat!

micha20:06:50

the -BP tells boot not to evaluate your build.boot or profile.boot files

micha20:06:10

if you leave the -BP off you will see the exception being thrown again

micha20:06:18

indicating it's something in one of those

richiardiandrea20:06:19

@micha for you, a dump of the objects after a couple of boot watch test iterations...

richiardiandrea20:06:56

I am going down the rabbit hole

micha20:06:15

iirc there is a way in there to see which objects are holding a reference to them

richiardiandrea21:06:28

there is ClojureRuntimeShimImpl but I guess I need to checkwhere the column "retained" is max?

micha21:06:37

the classloaders probably can't be GC because something is holdig a reference to a class created by them

micha21:06:56

i wonder if we need to call shutdown-agents

micha21:06:07

in the pod

micha21:06:22

maybe we already do actually

micha21:06:14

hm we do not

micha21:06:18

perhaps adding a line to eval (clojure.core/shutdown-agents) in the pod here: https://github.com/boot-clj/boot/blob/master/boot/pod/src/boot/pod.clj#L826

micha21:06:26

are you doing a parallel multi-build thing there btw?

micha21:06:41

passing data when you craete pods?

micha21:06:53

because that could result in pods not being eligible for GC

micha21:06:12

i mean possibly

richiardiandrea21:06:48

@micha no no multi, it is plain boot-test

richiardiandrea21:06:35

but I have to check that I don't create unnecessary pods yes

micha22:06:00

very strange

micha22:06:10

which version of clojure?

micha22:06:14

that could be a factor

micha22:06:35

the shutdown-agents thing didn't help?

richiardiandrea22:06:36

I am going to try now 😉

richiardiandrea22:06:02

clojure version 1.8.0

richiardiandrea22:06:53

it looks better with shutdown-agents

micha22:06:07

yeah that looks pretty good

micha22:06:19

it's clearing things out it seems

micha22:06:50

awesome! thanks

richiardiandrea22:06:00

no thank you Micha 😄

micha22:06:13

sorry you spent so much time on this

micha22:06:29

it's much appreciated 🙂

richiardiandrea22:06:42

I can say the same ah ah

richiardiandrea22:06:56

@micha, just to confirm the position:

(when pod
     (.close pod)
+    (clojure.core/shutdown-agents)
     (.. pod getClassLoader close)))

micha22:06:34

i was thinking to run that in the pod

micha22:06:56

(with-eval-in pod (clojure.core/shutdown-agents))

micha22:06:08

before .close probably

micha22:06:36

i fully qualified the name because the user could be shadowing shutdown-agents in the pod

richiardiandrea22:06:18

what was the intuition behind the shutdown-agents btw? each pod bootstraps his own set of agents (I guess) ?

richiardiandrea22:06:49

yes same seesaw pattern, good!

richiardiandrea22:06:08

ok ready to roll 😉

martinklepsch22:06:14

👏 @richiardiandrea ... memory leaks, always so fun to debug 😄

micha23:06:49

@richiardiandrea: you mentioned that the out of memory condition happened in an agent thread pool, and i remembered that those need to be shut down manually

richiardiandrea23:06:26

yeah it was executing code in a thread from an agent pool, who put it there I don't know though, maybe it is part of the pod-pool implementation but did not check that

kenny23:06:39

Is it possible to have two myjs.cljs.edns, one for dev and one for prod?

micha23:06:34

@kenny: yes, you can put them in different source directories and include one when building for development and the other when you're building for production

micha23:06:50

however you can also use macros to accomplish what you want probably

micha23:06:57

with one single entry point

micha23:06:23

you can make a macro that emits one thing when the ENVIRONMENT env var is "dev" and something else when it's "prod"

kenny23:06:49

Right. Still need the two directories, correct?

micha23:06:59

no, just one file

micha23:06:28

like the way i do environment-specific things in my cljs applications is to use macros

micha23:06:43

in a macro you can read environment variables

micha23:06:47

in the build environment

micha23:06:57

and the macro can emit different things depending on that

anmonteiro23:06:09

what’s the equivalent to Lein’s :java-source-paths in Boot?

micha23:06:23

@anmonteiro: just regular :source-paths

anmonteiro23:06:31

hrm, doesn’t seem to detect my Java class

micha23:06:33

boot doesn't know anything about clojure or java or anything

micha23:06:43

what's the symptom?

anmonteiro23:06:03

I might be doing something wrong, atm just java.lang.ClassNotFoundException

anmonteiro23:06:08

let me try with a dummy one

kenny23:06:17

I was wanting to customized the compiler options depending on dev or prod. Wouldn't the macro need to modify the file?

kenny23:06:21

Also, I found this article: https://www.martinklepsch.org/posts/parameterizing-clojurescript-builds.html. Couldn't you use that method instead of macros?

anmonteiro23:06:31

@micha: yeah, doesn’t seem to find it

martinklepsch23:06:47

@kenny: compiler-options itself can also be directly passed to the task

micha23:06:53

@kenny: yep, totally

anmonteiro23:06:55

package foo;

public class Foo {
  public static final FOO = 42;
}

anmonteiro23:06:04

under src/foo/Foo.java

anmonteiro23:06:37

(import [foo Foo])
;; => java.lang.ClassNotFoundException: foo.Foo

micha23:06:49

did you compile the Foo class?

anmonteiro23:06:02

How can I expect it to work?

anmonteiro23:06:06

sorry for the noise

kenny23:06:06

Oh really? I could've sworn I tried that before and it didn't work. Have you always been able to do that?

micha23:06:09

does the .class file show up in the fileset?

anmonteiro23:06:22

it had to be a stupid thing

micha23:06:29

ah ok 🙂

anmonteiro23:06:36

let me try to actually compile the thing

micha23:06:47

the javac task can do it

anmonteiro23:06:03

should I hook it up into the build pipeline somehow?

micha23:06:32

you can just do like boot javac repl

micha23:06:34

for instance

micha23:06:40

and it should compile your class

anmonteiro23:06:38

@micha: works as expected. My ignorance got in the way

micha23:06:54

@anmonteiro: glad it's working now 🙂

anmonteiro23:06:07

@micha: how should I pass JVM options to the javac task?

anmonteiro23:06:49

Boot’s source says it should be a [str]

kenny23:06:06

Are the options set in a cljs.edn file merged with the options set in the cljs task?

micha23:06:19

@anmonteiro: that means vector of strings

martinklepsch23:06:04

@kenny: to some degree probably yes, not sure about the contents of compiler options etc

anmonteiro23:06:23

@micha: yeah I got that, just wondering how to pass a vector at the command line

anmonteiro23:06:32

should I just -o xxx -o yyy ?

kenny23:06:10

It'd be nice to create a template with options and have the options in a cljs.edn override options set in the task.

micha23:06:14

the task help thing with --help will say something like Conj NAME onto the vector of javac options to use. or something like that

micha23:06:07

@kenny: the benefit of the .cljs.edn way is that the options are open rather than closed

micha23:06:17

like other tasks can modify them, look at them, etc

micha23:06:36

without needing to know anything about the specifics of the boot-cljs task even

micha23:06:59

that file is more of a specification, like the ring request/response maps