Fork me on GitHub
#boot
<
2016-10-12
>
lmergen08:10:44

seems to me that i’m running into an issue where the environment is loaded twice, it seems

lmergen08:10:59

this only seems to happen when i invoke boot test

lmergen08:10:13

http://pastie.org/private/7tkkhiuw8sc7buchkhmwaa shows what i see after invoking boot test — you can clearly see it is invoking al the logging initializations twice, etc

lmergen08:10:30

how would i go around debugging this ?

lmergen08:10:01

it seems as if there are two instances running, one being my application, and another being the instance boot is running in

dm309:10:55

can you paste the build.boot?

martinklepsch09:10:50

@lmergen boot-test runs your application in a pod (an isolated runtime of sorts) which may cause startup outputs to be printed twice

anmonteiro09:10:49

@danielsz sounds cool, but I’m not sure I understand the differences/implications

anmonteiro09:10:51

can you elaborate?

balupton11:10:47

Hey, been trying to get a Docker container setup for boot but I’m new to both clojurescript and docker! Does anyone know have any pointers to get started - there are a few clojurescript docker containers but they don’t seem well explained or maintained, and none for boot

martinklepsch11:10:13

@balupton could you outline what you expect the container to handle specifically?

balupton11:10:23

installation of java, clojure, clojurescript - then attachment of a clojurescript projects directory to the docker maintainer - such that the project’s team doesn’t need to all maintain or share their own vmware virtual machines

martinklepsch11:10:40

@balupton so you're looking for a development setup if I understand that right?

balupton11:10:24

that is correct - although the GUI editor and everything would of course happen on the host machine outside of the docker container

martinklepsch11:10:13

I don't quite see why you'd want to do that tbh. In any case have you used Boot to compile ClojureScript before?

balupton11:10:52

No I haven’t complete newb for clojurescript/docker

balupton11:10:38

okay I just found https://github.com/adzerk-oss/boot-clj-docker-image and https://github.com/kozily/docker-cljs/blob/master/Dockerfile - seems the trick is to search github for docker images rather than searching docker’s hub for docker images

balupton11:10:44

will experiment with that a little bit

balupton11:10:01

as I think perhaps I should try and get a basic setup going before attempting to get a project onto it

martinklepsch11:10:03

yeah, try understanding how boot works and then make it fit into a container. As Java + Boot/Leiningen are the only dependencies for Clojure/Script development I don't see a big benefit in putting it into a container for development but I don't know your situation 🙂

balupton12:10:01

cool thanks for your help, makes sense 🙂

lmergen12:10:27

@martinklepsch aha, i think that pod issue might indeed be the case. is it correct that not every task is executed in such a pod ?

lmergen12:10:35

for example, ‘boot repl’ doesn’t show this behaviour

lmergen12:10:04

anyway, in my case, the AWS SDK is being loaded twice, which causes it to register for javax metrics twice, which throws an error the second time

martinklepsch12:10:56

@lmergen pods are a boot features tasks may use or not

martinklepsch12:10:02

so yes not every task uses pods

lmergen12:10:16

right, then i suspect this is the issue.

dominicm12:10:25

Maybe boot alt test is what you want

lmergen12:10:16

didn’t know about that, it’s worth a shot

lmergen12:10:18

let me try

lmergen12:10:36

nop, same behavior

danielsz12:10:16

@anmonteiro Sure, thanks for asking. The difference between standard Lisp mode and tools.namespace mode is that standard Lisp mode doesn't pretend to "unload" code. It recompiles your namespaces in your project, with new code overwriting old code. Tools.namespace uses remove-ns to "remove" old definitions first, which in theory is good, but in practice not, because all remove-ns does is a unmapping, with old definitions continuing to linger in memory. If you're interested in this subject, I wrote a blog post that goes in detail, and I gave a talk around this at ICFP recently. With standard Lisp mode, your REPL behaves like all Lisp and Scheme REPLs, where you "recompile until it breaks". Does that make sense?

anmonteiro12:10:08

@danielsz oh alright, I get it yeah

anmonteiro12:10:23

I’ve read your post and the c.t.n. README so it makes sense

danielsz12:10:12

anmonteiro Cool. I switched to Lisp mode, I prefer it. It's less troublesome and you don't need to restart as often as with tools.namespaces. You should try it.

juhoteperi12:10:41

Hmph, I wonder if you are being hit by some c.t.n bug if you need to often restart REPL (when letting ctn to remove ns)

juhoteperi12:10:22

I haven't had any problems since I started using ctn fork with tns-45 fix

danielsz13:10:34

@juhoteperi No, sorry I didn't mean restarting the REPL, I meant restarting the application. With tools.namespace, you need to restart all threads everytime "remove-ns" is called, because old Vars remain trapped in threads. With standard Lisp mode this doesn't happen, so you can just recompile and continue working.

juhoteperi13:10:14

Aha. I anyway call reset quite rarely, like when I have edited several files. Often I just require the changed ns from editor.

danielsz13:10:08

Yes, that is the standard way of doing things in Lisp REPLS, you just recompile the forms you changed.

danielsz13:10:11

And beware of refresh, it doesn't quite do what it promises.

juhoteperi13:10:03

What would be the danger of old defs lingering in memory? At least it removes vars from ns-publics and that is important in our projects

danielsz13:10:55

@juhoteperi Good question, and you've put the finger on the benefits of using remove-ns, because it is indeed useful to clear the Vars.

juhoteperi13:10:56

The blog post seems to answer this

borkdude13:10:17

is it possible to have a piece of code executed in a library without requiring anything?

danielsz13:10:00

@juhoteperi Yes, no real danger, it's a tradeoff.

borkdude13:10:17

@danielsz well, it would be for something like https://github.com/borkdude/boot.bundle.edn - when loading it, it could merge some data in a map. It could also be done on require, but I just wondered if such a thing is at all possible without it

borkdude13:10:43

maybe when using AOT?

alandipert14:10:58

@borkdude can you elaborate on what you want?

borkdude15:10:39

@alandipert I dismissed the idea already, but it was something like this: in build.boot:

(set-env!
 :dependencies '[[boot-bundle "0.1.0-SNAPSHOT" :scope "test"]
                 [nl.michielborkent/boot.bundle.edn
                  "0.1.0-SNAPSHOT"
                  :scope "test"]])
then the second library automatically merges something in a map defined in the first.

borkdude15:10:41

being explicit is probably better

donaldball15:10:13

Morning boot friends. Having finished writing my aws-cli-wagon to take an end run around dependency hell, I tried using bootlaces to push it to clojars. Unfortunately, it fails:

donaldball15:10:20

~/src/aws-cli-wagon[master]$ boot push-release -f target/aws-cli-wagon.jar

donaldball15:10:35

Could not sign target/aws-cli-wagon.jar
gpg: can't open `target/aws-cli-wagon.jar': No such file or directory
gpg: signing failed: No such file or directory

donaldball15:10:44

The file does exist 😕

donaldball15:10:28

And I can install via boot install -f target/aws-cli-wagon.jar

alandipert15:10:36

how about boot build-jar push-release?

alandipert15:10:57

@borkdude i think you might be able to do what you might want by abusing export-tasks meta on a clj namespace

donaldball15:10:07

The build-jar task doesn’t do the aot I need

donaldball15:10:08

(deftask build
  []
  (comp (aot) (pom) (jar) (target)))

donaldball15:10:33

I suppose I could replace target with push-release in a separate bespoke push-my-release task?

alandipert15:10:56

maybe boot aot build-jar push-release?

alandipert15:10:08

i'm surprised to (re)discover that bootlaces needs target at all

alandipert15:10:19

i guess it's been a couple years and target has been deprecated since we made it

donaldball15:10:30

I don’t know that it does. I’m surprised bootlaces’ push-release doesn’t work with a specified path

alandipert15:10:41

yeah, me too

donaldball15:10:41

Or at least that it’s inconsistent with install

donaldball15:10:48

Oh, is target deprecated?

alandipert15:10:08

err not the target task, but the default behavior of writing to target always

alandipert15:10:19

target task is the new and good way to doit

donaldball15:10:40

cool, that was my impression

alandipert15:10:07

i guess the push task must be doing something magical

alandipert15:10:15

since push-release sends it a nil file arg

donaldball15:10:33

Okay, boot aot build-jar target gives me the jar file I want

alandipert15:10:35

then in the task body it does some sniffing by .jar extension if file i nil

alandipert15:10:34

when you do boot aot build-jar show -f are there jars in the fileset?

donaldball15:10:46

but boot aot build-jar push-release fails ha ha

donaldball15:10:58

clojure.lang.ExceptionInfo: Assert failed: project repo is not clean
                            (or (not ensure-clean) clean?)

alandipert15:10:21

it refuses to do it if git is in dirty state

donaldball15:10:31

~/src/aws-cli-wagon[master]$ boot aot build-jar show -f
Compiling 1/1 sparkfund.maven.wagon.aws-cli...
Writing pom.xml and pom.properties...
Writing aws-cli-wagon.jar...
Installing aws-cli-wagon.jar...

├── leiningen
│   └── wagons.clj
├── META-INF
│   └── maven
│       └── sparkfund
│           └── aws-cli-wagon
│               ├── pom.properties
│               └── pom.xml
├── aws-cli-wagon.jar
└── sparkfund
    └── maven
        └── wagon
            ├── aws_cli$_closeConnection.class
            ├── aws_cli$_get$fn__75.class
            ├── aws_cli$_get.class
            ├── aws_cli$_getIfNewer.class
            ├── aws_cli$_init.class
            ├── aws_cli$_openConnectionInternal.class
            ├── aws_cli$_put.class
            ├── aws_cli$fn__66.class
            ├── aws_cli$loading__5340__auto____64.class
            ├── aws_cli.class
            ├── aws_cli.clj
            └── aws_cli__init.class

donaldball15:10:41

ha ha c’mon boot, you’re not the boss of my git state 😛

alandipert15:10:54

well boot isn't, but this bootlaces thing

alandipert15:10:56

is a bit overbearing

donaldball15:10:57

cool, boot aot build-jar push-release does the trick

donaldball15:10:05

With a clean git

donaldball15:10:54

And now I have a private s3 wagon that works with boot and amazon sts credentials

hiredman16:10:28

I am using boot to build what is more or less a multimodule build, a bunch of subprojects together with a single build system. I have a boot task which tests a given subproject in isolation, meaning it doesn't load the other subprojects stuff in to the environment. I would now like to write a test-all-subprojects boot task, which would run each subproject's tests in isolation. It seems like pods would be the way to do this, but...

hiredman16:10:04

1. It doesn't seem like I can run a boot task in a prod

micha16:10:07

you can run boot itself in a pod from your boot script

hiredman16:10:18

2. The existing testing framework's boot task already uses a pod to run the tests, which gets its environment with get-env

micha16:10:47

that example has dependencies between the modules, which you may or may not have in your own project

micha16:10:58

if you don't have dependencies you can probably forgo the semaphores

micha16:10:13

so that example consists of two modules, alpha and bravo

micha16:10:18

bravo depends on alpha

micha16:10:30

one build.boot file

micha16:10:38

there are 3 tasks in there that make it work

hiredman16:10:41

that boot in boot looks like exactly what I need

micha16:10:55

one to build alpha, one to build bravo, and the doit task to coordinate launching the other two

micha16:10:37

that's the dependency linkage

micha16:10:05

alpha is built and its artifacts are emitted to a directory that is on the classpath for bravo's build

micha16:10:25

the doit task there should work with watch etc

micha16:10:50

and @richiardiandrea is the resident parallel boot build expert 🙂

hiredman16:10:40

I wrote some code that scans the subprojects using tools.namespace and figures out the dependencies between them

flyboarder16:10:40

@micha: is there a way to get the root project directory without passing it around?

flyboarder16:10:49

When using multi-boot

micha16:10:10

@flyboarder how do you mean "root project directory"?

flyboarder16:10:25

The directory containing build.boot

micha16:10:36

it should be . no?

flyboarder16:10:22

I have a user who reported an error on one of my tasks seems that path is different when they use multi-boot

flyboarder16:10:39

I'll have to dig more I think

micha16:10:49

i believe that (io/file ".") is the same anywhere in the JVM

micha16:10:57

pods or boot-in-boot can't change that

flyboarder17:10:13

Ok, it's probably not the path itself then

micha17:10:24

that's one reason why it's difficult to make a resident boot server

flyboarder17:10:31

Maybe some inferring going on

micha17:10:46

you can do this: (.getCanonicalPath (java.io.File. "."))

micha17:10:01

that should return the same string, JVM-wide

flyboarder17:10:13

Cool thanks!

arohner17:10:22

checkout says it’s deprecated, so what is the new way to do that?

arohner17:10:48

the changelog mentions —checkouts, but I’m not finding any docs on it

micha17:10:43

@arohner you can do like this:

micha17:10:17

boot --checkouts foo/bar:1.2.3 watch build

micha17:10:40

you will need to have a foo/bar dependency in your :dependencies in your build.boot

micha17:10:53

but it can be a different version than the one you're checking out

arohner17:10:13

and I still have to boot/lein install foo/bar as I work on it?

micha17:10:18

you can have multiple --checkouts ... options

micha17:10:33

what i do is have a separate boot running with watch

micha17:10:48

the builds will cascade sort of

micha17:10:08

if you have watch in the build with --checkouts

micha17:10:27

that decouples the builds

arohner17:10:35

my normal process is to boot repl. watch repl is not a good idea, because it would keep restarting, right?

micha17:10:35

so the --checkouts one simply consumes a jar

micha17:10:58

you can do (def p (future (boot (watch) ...))) in the repl

micha17:10:10

then if you want to stop it you can do @p and ctrl-c

micha17:10:25

there is also a pretty cool library from gfredricks

micha17:10:34

for managing background tasks in the repl

micha17:10:22

or, alternatively you can do boot watch repl -s ...

micha17:10:32

and then connect from a separate client or cider or whatever

micha17:10:11

that just starts the repl server so you don't need to block the pipeline

micha17:10:22

that way the watch task can do its thing

arohner17:10:08

why is the watch necessary? If I just boot —checkouts … repl, what happens?

micha17:10:06

that should work, yes

micha17:10:30

and you can run boot with watch from inside the repl if you want to

micha17:10:40

or you can just reload namespaces in the repl

micha17:10:50

the claspath will update when the checkout dep changes

donaldball17:10:17

It seems to be idiomatic boot to wrap project var names with +, er, pauldrons. Why is that? Just for visual distinctiveness or is there some magic of which I’m as yet unaware?

micha17:10:58

@donaldball no magic, just silliness

micha17:10:27

i like them when i have some hardcoded constant that i will update manually

micha17:10:36

like the project version

martinklepsch17:10:44

@donaldball I think it's a convention @micha came up with and many just adopted without much thought

danielsz18:10:15

@donaldball Earmuffs for * and, er, pauldrons for +? Really? Interesting. Please share the etymology for that word if you can.

hiredman18:10:39

so I have runboot working pretty nice, except it seems like runBoot returns before clean up actions have been run. I haven't teased out exactly what is happening but when I use runboot to run tests for two subprojects (call them subprojects A and B) I get clean test runs for A and B, but an exception in middle about not being able to load the namespace that subproject A loads as part of initializing the pod where tests are run.

donaldball18:10:08

(Pauldrons are the shoulder parts of some kinds of armor)

hiredman18:10:22

the test framework uses a pool of pods and registers a clean action to shutdown the pool

hiredman18:10:58

if I add a wait after the runboot, the stacktrace in the middle goes away

hiredman18:10:37

the test framework calls refresh on the pool after it finishes running the tests

hiredman18:10:48

so it looks sort of like the tests finish running, the refresh starts, then runboot exits, then a new runboot starts, which changes the environment, which somehow effects the refresh of that other pod, which causes it to throw an error in the background

richiardiandrea18:10:43

@hiredman that makes sense to me and to be honest I don't even know if it's worth handling it...there was a discussion here with @micha about tests in a pod and how slow it can be to load the symbols in a fresh env every time...

hiredman18:10:04

it seems like async tasks started while executing boot.App/runBoot should finish or be stopped before runBoot returns, no?

micha18:10:37

@hiredman some kind of synchronization might be needed

hiredman18:10:57

I am not running the tests in parallel

micha18:10:15

i think one difference between boot-in-boot and boot is that the JVM will wait for certain things before exiting, but boot-in-boot isn't asking the JVM to exit

micha18:10:45

are you using the semaphore technique?

hiredman18:10:03

in breaks the abstraction, because I just can't go calling tasks willy nilly in runboot then

hiredman18:10:09

I am not running tasks in parallel

hiredman18:10:27

I build a (comp (runboot ...) ... (runbot ...))

micha18:10:04

those will run in parallel unless you have some kind of barrier

hiredman18:10:05

and I removed the future stuff inside runboot

micha18:10:45

so yes, tasks should return before runboot returns

richiardiandrea18:10:10

yes, there is a explicit future somewhere there which is kind of "unexpected"

hiredman18:10:20

and they do, but it seems like some pod related stuff is left outstanding, specifically, I think, the pod being built as a result of the refresh called by the testing framework's task, is still being built after runBoot returns, and some how (this seems really weird to me, but I am not familiar with boot internals) it is still trying to init with the requires setup from the previous tests, but doesn't have the environment for those requires to work

hiredman18:10:58

maybe somewhere the pooled pods are recalculating their environment / classpath when refreshed instead of re-using the previous environment in the pool

micha18:10:25

pod pools give you a fresh pod each time you :refresh

micha18:10:10

the :refresh thing is there to let the pool handle destruction of the pod

micha18:10:15

but you don't need to use that

micha18:10:25

you can use um, :take i think it is

hiredman18:10:41

sure, I am not, the test framework (not my code) is calling refresh

hiredman18:10:58

I think I will just monkey patch the testing framework to not use a pool

danielsz19:10:33

@donaldball Yes, but I was wondering how the term became associated with +. Is this your invention, or have you seen this somewhere?

donaldball19:10:41

Oh, my own invention

richiardiandrea20:10:55

so @micha you are saying it is not worth working on the line number feature because of cat -n right? 🙂

dominicm20:10:17

You never know what they do until someone points out that your monstrosity is a one-liner in bash

dominicm20:10:21

Then you're still unsure

richiardiandrea20:10:39

that is why it might make sense to include them I would say, but I am asking first of course ...it is 5 lines of code more or less

sekao20:10:18

i don’t suppose anyone knows why boot would completely ignore the src directory. it just started happening today. i can literally move all the files out of it and the application still runs. i feel like i’m going crazy...

richiardiandrea20:10:57

@sekao there is a lot of caching going on in ~/.boot/, this is not entirely surprising , even though I never happened to try that 😄

sekao20:10:13

thanks @richiardiandrea you pointed me in the right direction. i just remembered that earlier today I started pulling the deployed version of my project in my profile.boot. so when i started developing again, the deployed version was still being pulled and used 😂

richiardiandrea20:10:55

np probz, yeah profile.boot sometimes gets in the way indeed 😄

micha21:10:18

@richiardiandrea i would prefer not to add features that are so natural to do separately in the shell

richiardiandrea21:10:16

Ok @micha, fair enough 😉