Fork me on GitHub
#boot
<
2016-03-26
>
jamesleonis01:03:25

@lilymgoh: This might be off, but are you using boot in a folder with a build.boot file?

lilymgoh02:03:54

It erred in the root directory as well

jamesleonis02:03:29

What version of Java are you running?

jamesleonis02:03:56

So far the same as my setup...

jamesleonis02:03:53

I just tried uninstalling it from homebrew, blowing away my m2 and .boot directory, and reinstalling it. So far it works.

lilymgoh02:03:07

Yup, I did that as well

lilymgoh02:03:38

Also tried restarting, that didn’t work as well 😞

jamesleonis02:03:46

Instead of installing it through homebrew, have you tried the manual install method? https://github.com/boot-clj/boot#unix-linux-osx

lilymgoh02:03:50

I wonder if that’s the problem though, but I’ll give it a shot

jamesleonis02:03:53

I don’t think so, but the obvious suspects are innocent here

lilymgoh02:03:01

Jarusvasunt@LilyJ ~> brew uninstall boot-clj
Uninstalling /usr/local/Cellar/boot-clj/2.5.2... (2 files, 6.6K)
Jarusvasunt@LilyJ ~> sudo bash -c "cd /usr/local/bin && curl -fsSLo boot  && chmod 755 boot"
Password:
Jarusvasunt@LilyJ ~> boot
clojure.lang.ExceptionInfo: Unable to resolve symbol:  in this context
    data: {:file
           "/var/folders/z_/ffgkd2vx1njf0bgs8vvngn600000gn/T/boot.user9003804281974570463.clj",
           :line 0}
java.lang.RuntimeException: Unable to resolve symbol:  in this context
               ...                
boot.main/-main/fn   main.clj: 192
   boot.main/-main   main.clj: 192
               ...                
  boot.App.runBoot   App.java: 390
     boot.App.main   App.java: 467
               ...                
         Boot.main  Boot.java: 258
Jarusvasunt@LilyJ ~ [1]> 

lilymgoh02:03:07

exact same error 😞

lilymgoh02:03:01

I’m on OSX 10.11.4, btw

jamesleonis02:03:59

I’m on 11.3. I’m going to update and see if that is the problem

jamesleonis03:03:47

Welp, still works with 11.4 on my box

lilymgoh03:03:55

Oh well, thanks anyway. I'll try more later.

malcolmsparks09:03:32

I have a couple of feedback points about boot I have configured my profile.boot with this

(configure-repositories!
 (let [creds-file (File. (boot.App/bootdir) "credentials.gpg")
       creds-data (gpg-decrypt creds-file :as :edn)]
   (fn [{:keys [url] :as repo-map}]
     (merge repo-map (creds-data url)))))
as per https://github.com/boot-clj/boot/wiki/Repository-Credentials-and-Deploying

malcolmsparks09:03:55

However, it gets called everytime I use boot, which is a bit of a pain for me because it involves me unlocking my gpg private key each time - whereas with lein it was only when I was actually deploying to clojars (and not the rest of the time, like when I'm offline)

malcolmsparks09:03:24

The other feedback point is that I've ended up with

(push :file "target/mylib-1.0.0-SNAPSHOT.jar")

malcolmsparks09:03:58

It seems there should be a way of 'deriving' the target file without hardcoding the project name and version - it feels redundant to me - is there a better way?

malcolmsparks10:03:56

It would be somewhat nicer in boot if the repo-map could be lazy, such that the credentials were only sought once it a repo has been determined

isaac10:03:02

WARNING: Unknown compiler option ':warning-handlers’
clojure
                 [adzerk/boot-cljs              "1.7.228-1"        :scope  "test”]
                 [org.clojure/clojurescript      "1.8.34"]

boot-cljs warning on use clojurescript 1.8

juhoteperi10:03:24

@isaac: Bug in clojurescript, will be fixed in next release

isaac10:03:53

Should I use clojurescript 1.8 now?

juhoteperi10:03:27

The warning doesn't cause problems, you can use it if you want. If you don't want to see the warning keep using 1.7.

juhoteperi10:03:21

Boot-cljs should be mostly version agnostic nowadays, I will probably change the version scheme in future.

isaac10:03:33

That ok. the warning just confuse me. I will use 1.8 if no problem

malcolmsparks10:03:21

It would be nice if boot supported the automatic lookup of gpg credentials from a user's $HOME/.boot directory - I've had to jump through quite a few hoops to get something that is optimal. Do you think there would be much appetite for this if I created a PR?

juhoteperi10:03:48

Not sure what is the opinion making it more automatic, if I recall correctly it was decided to leave to use to select where to load the creds from

juhoteperi10:03:55

I have a problem with the current implementation, the GPG decrypt is called each time I open boot which means I would need to have yubikey always attached

malcolmsparks10:03:01

Yes, those are the instructions I followed. I think the decrypt on every run aspect of configure-repositories! in that wiki is a little severe though - and there should be a sensible default that works for most users

juhoteperi10:03:16

Lein only calls decrypt when it needs creds, i.e. when doing deploy

malcolmsparks10:03:54

Yes, I have the very same problem! I have a yubikey on my keyring, and I don't want to keep it plugged in for normal 'offline' development, I only want to plug it in when i do crypto, and take it out when I'm done

malcolmsparks10:03:41

for what it's worth, he's what I'm doing in my profile.boot

;; Get credentials from a GPG encrypted file.
(defn decrypt-creds [url]
  (let [creds-file (java.io.File. (boot.App/bootdir) "credentials.gpg")
        creds-data (boot.core/gpg-decrypt creds-file :as :edn)]
    (some #(when (re-seq (first %) url) (second %)) creds-data)))

(defn repo-map [repo]
  (case repo
    "clojars" (let [url ""]
                (merge {:url url}
                       (decrypt-creds url)))
    (throw (ex-info "Unknown repo" {:repo repo}))))

malcolmsparks10:03:21

Then I call (repo-map "clojars") in my deploy task - e.g.

(push :repo-map (repo-map "clojars"))

malcolmsparks10:03:38

obviously that's suboptimal because it requires someone's got the same set up as me in their profile.boot

juhoteperi10:03:55

I have just decrypted the creds file

malcolmsparks10:03:48

The decrypt-creds is more complex because I put regexes in my credentials.gpg

malcolmsparks10:03:19

But there's nothing in my profile.boot that couldn't reasonably go into boot.core, imho

malcolmsparks10:03:56

as a 'reasonable' default, you should always have the option of providing your own logic in profile.boot, but it should be for escaping the defaults, not simply trying to follow the happy path

malcolmsparks10:03:19

and gpg encryption of creds should arguably be the default - because security simple_smile

juhoteperi10:03:50

I think before we try to decide the reasonable defaults we should create a way to decrypt deploy creds only when needed

juhoteperi10:03:16

maybe something like configure-deploy-repositories!

juhoteperi10:03:15

Yeah GPG encryption is reasonable default, but I'm not sure how important it is for people who already have encrypted disk

malcolmsparks10:03:21

I agree. By the way, that's what my repo-map function is for - it's lazy in that I only call it when it's needed. It's not a great solution but something that is lazy in boot would be good

malcolmsparks10:03:35

An encrypted disk offers no protection at all if your computer is on

malcolmsparks10:03:51

Mine usually is!

juhoteperi10:03:23

Should be quite hard to access the files if computer is locked

juhoteperi10:03:25

or suspended

malcolmsparks10:03:44

And since lein has already established this as a common practice, there's lots of resources out there about how to set it up.

malcolmsparks10:03:03

I still wouldn't put all my passwords in a clear-text file - I'm happy that they're in credentials.gpg - and if I accidentally copy that file somewhere I don't have to worry about it ending up at github

malcolmsparks10:03:46

And my hard disk is encrypted

malcolmsparks10:03:59

But anyway, something should be done to improve the overall experience of setting up a project to push a simple library to clojars - it took me a lot longer than I had expected

juhoteperi10:03:30

Issue describing the problems should be a good start

juhoteperi10:03:52

Or maybe two issues

juhoteperi10:03:07

1. decrypt deploy creds only when needed, 2. better overall experience

juhoteperi11:03:14

@micha: I think the current "2.6.0-SNAPSHOT" development model is quite confusing, especially if the branch is rebased against master. I think it would be better to have default branch (master or develop) which would have the most up-to-date code and PR's would be merged into this. If we need to make bugfix releases to current minor version, we could have separate branches for that.

micha12:03:40

@juhoteperi the 2.6 branch isn't rebased...what is confusing there?

juhoteperi12:03:29

PR's are made against master branch but some include 2.6 commits

juhoteperi12:03:31

so the diff is a mess

micha12:03:15

yeah, pr against master is a mess

juhoteperi12:03:51

Sure, PR's could be made against different branch, but it should be documented at least

juhoteperi12:03:32

And I think it would most obvious that the default branch is where development for next release happens, and that is the target for PRs

micha12:03:45

ok yeah if you want to merge the 2.6 branch and use the method you described going forward i think that would be good

juhoteperi12:03:06

Ok, I'll do that. There is one or two simple PRs I would like to merge also.

micha12:03:16

this will also keep the github crawler bots happy with our progress lol

juhoteperi12:03:49

I think maybe 2.6.0 branch was rebased since that was created?

micha12:03:07

i don't think that branch was ever rebased

juhoteperi12:03:22

Hmm, yeah all the commits have same sha

micha12:03:46

i think we made that branch because of the issues with rebasing

juhoteperi12:03:15

Manual merge worked as expected

micha12:03:01

i am pretty stoked to have tests

juhoteperi12:03:21

I think that should take care of trivial PRs. I think several others are fine also (tests, filter by metadata/properties, compile path)

juhoteperi12:03:25

Not sure about option docstring formatting.

micha12:03:41

you have reservations about that one?

juhoteperi12:03:57

I think the current format looks a bit better, maybe better fix for cases where it doesn't fit would be to allow completely skipping formatting?

juhoteperi12:03:07

^:raw "This option is not formatted"

juhoteperi12:03:28

... except strings can't have metadata 😞

micha12:03:13

i think it would have been better maybe to have designed that part to accept c style % directives

micha12:03:34

rather than trying to construct the docstring in the macro completely

juhoteperi12:03:10

Yeah something like that.

micha12:03:18

i'd really like to eventually have boot emit man pages

micha12:03:29

so you have plenty of space to describe things

micha12:03:43

like in cli mode at least

micha12:03:06

i looked everywhere though and could not find a java troff implementation

micha12:03:15

so windows would be a problem

micha12:03:23

yeah i think you're right, we could leave it as it is

micha12:03:38

it's quirky, but at least the logic is simple and consistent

micha12:03:48

so it's not hard to get used to the quirks

micha12:03:07

changing things might just aggrevate the problem

juhoteperi12:03:13

^ any ideas how easy it would be to read properties files in Bash? Maybe just using source as properties files should be valid shell scripts?

micha12:03:43

that would be hard to support in windows

juhoteperi12:03:58

windows has separate wrapper anyways?

micha12:03:05

there is no script in windows

micha12:03:09

it's an exe

micha12:03:04

i guess it's ok maybe because there is an ini file you edit in windows

micha12:03:01

yeah we could do it in bash though, no problem

micha12:03:21

i don't think we could just source the file because the values don't need to be quoted

micha12:03:42

java just splits on the first = i think

juhoteperi12:03:08

Probably it would be best to grab just those two specific keys from the file, so other variables are not accidentally set

micha12:03:48

we'd want to get the jvm options and the maven setup ones

micha12:03:10

and export them as env vars unless those env vars are already set

micha12:03:21

because env vars should override the files

juhoteperi12:03:22

What are the maven vars? LOCAL_REPO?

micha13:03:00

in 2.6.0 there is also BOOT_CLOJARS_REPO, BOOT_CLOJARS_MIRROR, BOOT_MAVEN_CENTRAL_REPO, and BOOT_MAVEN_CENTRAL_MIRROR

micha13:03:23

actually i don't think those need to be set in the env

micha13:03:31

because boot reads them before trying to fetch anything

micha13:03:38

from the file

micha13:03:58

maybe it's just BOOT_JVM_OPTIONS actually

micha13:03:24

was there another one that needed to be set in the environment?

micha13:03:40

ah BOOT_JAVA_COMMAND

micha13:03:02

although putting that in a file seems like a footgun to me

micha13:03:26

i guess we shouldn't judge though

micha13:03:42

people know what they're doing

micha13:03:35

i can add that to the boot-bin shim

micha13:03:53

i was also thinking about adding an option to boot

micha13:03:58

like currently we have this

micha13:03:02

-e --set-env KEY=VAL        Add KEY => VAL to project env map.

micha13:03:19

people have used this to implement "flavors" like this

micha13:03:30

boot -e flavor=dev ...

micha13:03:45

and in their task they can do (get-env :flavor)

micha13:03:09

but i think it's maybe not good to have the env be a data bag

micha13:03:33

so perhaps a separate map of key value options would be good?

micha13:03:21

the use case for flavors is mostly related to fetching dependencies

micha13:03:47

if you use a profile type task you end up fetching deps in two separate calls to set-env!

micha13:03:07

that causes problems with deps resolution

richiardiandrea17:03:10

Some git guru here maybe can tell me how I can create a patch that differs from master, from a certain commit to another

richiardiandrea17:03:36

the only think I can do, is to go to master and cherry pick merge my commits

richiardiandrea17:03:44

which I guess is fine too

juhoteperi17:03:51

Shouldn't be needed

juhoteperi17:03:04

I just pushed the exactly same branch to github and diff is okay: https://github.com/boot-clj/boot/compare/arichiardi-sift-fix-test

richiardiandrea17:03:32

oh let me try, without rebasing?

juhoteperi17:03:57

No rebase needed, history is the same. GitHub is just confused for some reason.

juhoteperi17:03:32

Yeah, that fixed it.

richiardiandrea17:03:15

so I will work on the thread issue, because if you have 1000 tests at the moment you also create 1000 threads

richiardiandrea17:03:24

then you can merge ok?

richiardiandrea17:03:08

now I rebased and everything is ok

richiardiandrea17:03:46

Do you think we should bound the number of threads? The tests are kind of short-lived

richiardiandrea17:03:20

I don't have enough experience in profiling this to take a decision

richiardiandrea17:03:26

the other option is to have the classic bounded pool like clojure actors' pooledExecutor:

Executors.newFixedThreadPool(2 + Runtime.getRuntime().availableProcessors(), 
		createThreadFactory("clojure-agent-send-pool-%d", sendThreadPoolCounter));

micha17:03:27

yeah using infinite threads probably not good

micha17:03:34

you can do them in batches though

richiardiandrea17:03:51

yes I was thinking about that

micha17:03:23

the simplest thing might be to make an executor pool

micha17:03:32

via the java stuff

richiardiandrea17:03:59

yes then I need to replace it for future calls

richiardiandrea17:03:08

or abandon future

micha17:03:20

does future take an executor as an optional argument?

richiardiandrea18:03:01

I could replace future with agent, which is more configurable

richiardiandrea18:03:03

using send actually I don't need to do anything else

micha18:03:33

sweet, yeah

richiardiandrea18:03:17

actually I do if I want it to be configurable, but for now I will just make it bounded

richiardiandrea18:03:41

bounded? lol don't know if it's correct English simple_smile

micha18:03:08

bounded, that's a thing simple_smile

richiardiandrea18:03:25

I have a weird error with the new version:

Assert failed: env :directories must be a set
                          (set? new)

micha18:03:10

are you setting :directories in any pod?

richiardiandrea18:03:23

mmm no that I am aware of

richiardiandrea18:03:40

investigating...

richiardiandrea18:03:50

and the input is a string, so conj creates probably a list

richiardiandrea18:03:18

the interesting question is why (get-env :directories) is nil

micha18:03:36

a change was merged today

micha18:03:42

that looks at boot.pod/env

micha18:03:48

instead of the atom

micha18:03:54

that's possibly the issue

micha18:03:16

does reverting that commit fix it?

micha18:03:52

i bet that introduces a race condition

richiardiandrea18:03:20

reverting g revert b55f444 -m 1

micha18:03:55

we can just revert that change for now

micha18:03:21

anyway if we're going to do that we'd want to do it completely and get rid of the atom

micha18:03:58

instead of just changing get-env

richiardiandrea19:03:00

ok then, if you want to do it cleanly I guess @juhoteperi can rebase or something

richiardiandrea19:03:17

that's good that we introduce testing 😉

micha19:03:19

we can have the revert commit in the history

richiardiandrea19:03:34

will push that on my branch

micha19:03:50

yeah regression testing is huge step forward

micha19:03:59

very exciting

juhoteperi19:03:13

@richiardiandrea: I'll push it to master, better than to include in non related branch

juhoteperi19:03:34

I thought I ran Saapas using that change but maybe it just didn't trigger the problem

juhoteperi19:03:38

Or maybe I forgot to build Boot after changes

richiardiandrea19:03:23

yeah the make install part is tricky, many times I forget that as well

micha19:03:19

i ran into problems working on a local snapshot version when there is a snapshot deployed to clojars

micha19:03:34

because of the way the different modules depend on each other

micha19:03:56

so for dev i'd set the version to 2.6.0

richiardiandrea20:03:00

the pool is not a good solution for boot in boot I reckon, for some reason I my threads are locked, maybe batching is easier to implement for now

richiardiandrea22:03:53

this warning for task redefinition is bothering me, can I ask you guys what was the concern when you introduced it?

richiardiandrea22:03:07

in boot in boot (and therefore parallel testing) of course tasks will always be redefined and it might make sense to disable it

alandipert22:03:56

i like it because it can highlight issues between profile.boot/build.boot

alandipert22:03:12

but i also would support an env var disabling it

richiardiandrea22:03:46

I was more thinking of util/*verbosity* but an env var is maybe a better idea

richiardiandrea22:03:53

now you English speakers should help me with a name for this var 😄

alandipert22:03:57

i don't know what -v does - is it hooked to a var?

alandipert22:03:09

i guess it must be, since some how util/info etc see it

richiardiandrea22:03:07

it is hooked to util/*verbosity*

alandipert22:03:15

maybe level 1 should print the task redef warnings?

micha22:03:39

with boot in boot you should have a fresh new boot.user namespace each time, so you shouldn't be seeing those warnings

richiardiandrea22:03:44

we start with level 1, and up, so maybe we don't print it by default (level 1) and we start printing it from level 2 (`-v`) ?

richiardiandrea22:03:13

it happens because we pass by -main I think:

clojure.core/require                                core.clj: 5832
                                 clojure.lang.RestFn.invoke                             RestFn.java:  421
                                          boot.user/eval108         boot.user852375673052143229.clj:   15
                                 clojure.lang.Compiler.eval                           Compiler.java: 6782
                                 clojure.lang.Compiler.load                           Compiler.java: 7227
                             clojure.lang.Compiler.loadFile                           Compiler.java: 7165
                                   clojure.lang.RT$3.invoke                                 RT.java:  319
                                         boot.main/-main/fn                                main.clj:  201
                                            boot.main/-main                                main.clj:  201

micha22:03:54

there should be a different boot.main too though

micha22:03:28

the newcore method should be making a new pod

richiardiandrea23:03:40

ok will try to follow the chain

richiardiandrea23:03:46

well the exception already gives me some hint:

clojure.core/require                                core.clj: 5832
                                 clojure.lang.RestFn.invoke                             RestFn.java:  421
                                          boot.user/eval108         boot.user538939168026557179.clj:   15
                                 clojure.lang.Compiler.eval                           Compiler.java: 6782
                                 clojure.lang.Compiler.load                           Compiler.java: 7227
                             clojure.lang.Compiler.loadFile                           Compiler.java: 7165
                                   clojure.lang.RT$3.invoke                                 RT.java:  319
                                         boot.main/-main/fn                                main.clj:  201
                                            boot.main/-main                                main.clj:  201
                                    clojure.lang.Var.invoke                                Var.java:  394
org.projectodd.shimdandy.impl.ClojureRuntimeShimImpl.invoke             ClojureRuntimeShimImpl.java:  159
org.projectodd.shimdandy.impl.ClojureRuntimeShimImpl.invoke             ClojureRuntimeShimImpl.java:  150
                                           boot.App.runBoot                                App.java:  399

micha23:03:42

maybe there is a bug in the way pods work?

richiardiandrea23:03:25

maybe, the thing is, the core pod always calls boot.main/-main in boot in boot: https://github.com/arichiardi/boot/blob/sift-fix-test/boot/base/src/main/java/boot/App.java#L398

richiardiandrea23:03:57

and this triggers the require

micha23:03:21

but you created that core pod, though, right?

micha23:03:44

so it should be a brand new boot.main/-main, a different one than the ones that ran before

micha23:03:02

my internet is ridiculously slow

micha23:03:02

how does this work? i don't see where runtests is coming from?

richiardiandrea23:03:04

for each command I have a runboot call

richiardiandrea23:03:44

if each runboot creates a separate pod and no bugs, you are right, there should not be any warning