Fork me on GitHub
#boot
<
2015-11-15
>
binduwavell02:11:18

I'm sorry this is likely a very dumb question. How can I configure the color of the return values in the boot repl. I assume the answer has something to do with nRepl, but I'm not quite sure where to look or if there is a boot specific way of setting this for my account no matter which project I'm working on. My background color is dark grey and the blue result color is exceedingly difficult to read.

micha02:11:37

@binduwavell: this annoys me, too

micha02:11:30

the best thing i can recommend is boot -C repl

micha02:11:05

the colors are configured by REPLy

micha02:11:27

(the -C option to boot disables colored output)

binduwavell02:11:16

Is there something like the whidbey lein plugin for boot?

binduwavell02:11:28

@micha that is way better for now, thank you!

micha02:11:31

you can use whidbey with boot i think

binduwavell02:11:36

Would be nice to have real color output though simple_smile

micha02:11:43

one second, i'll see if it works

binduwavell02:11:23

@micha can you point me at docs for how to use lein plugins with boot? I'm happy to give whidbey a shot.

micha02:11:41

you can't really use lein plugins with boot

micha02:11:48

you can ue the underlying functions etc

micha02:11:12

it's not clear to me exactly what the whidbey lein plugin is doing to nrepl and reply

micha02:11:27

but boot uses nrepl and reply too, so it should be possible

binduwavell02:11:12

Did you see the following from their readme: "Internally, Whidbey reaches into nREPL to replace the default pr-values rendering middleware. See the history for more on the motivations and implementation details behind this project."

micha02:11:02

you can add nrepl middleware to the boot repl task if that's what you need to do

binduwavell02:11:40

Thanks @micha I hope I can figure out how to get these things to work together, would be super nice! Appreciate the pointer to the Cider-REPL doc...

micha03:11:17

if you can make it work i'd enjoy seeing how you did it simple_smile

jaen10:11:27

I've just downloaded new boot.sh and it started doing

Downloading .
when launching boot. Is that expected?

juhoteperi10:11:20

@jaen: Does the project you are working with have a boot.properties file which defines BOOT_VERSION?

jaen10:11:39

No, this one doesn't.

jaen10:11:48

I didn't create it yet

jaen10:11:57

And this keeps repeating on each boot command, as if it couldn't download boot

juhoteperi10:11:19

Strange. If you don't have BOOT_VERSION defined it should use the latest version.

jaen10:11:03

This didn't happen before I downloaded new boot.sh if that helps

juhoteperi10:11:35

Perhaps you should clear the boot temp dir? (~/.boot/cache)

jaen10:11:41

Hm, when I added boot properties and specified 2.4.2 this stopped happening.

jaen10:11:10

But funny it kept trying to redownload it each time with 2.0.0

juhoteperi10:11:24

Probably there is a old ~/.boot/cache/boot.properties file which points to 2.0.0

juhoteperi10:11:36

But redownloading sounds like a bug

jaen10:11:50

The ~/.boot/cache/boot.properties points to 2.4.2 for me, but I imagine this was overwritten when I added the boot.properties to the project.

juhoteperi10:11:30

I don't think project boot.properties should have any effect on global boot.properties

juhoteperi10:11:02

What does boot -V say if you run it outside the project dir?

jaen10:11:35

Downloading .
#
#Sun Nov 15 11:47:21 CET 2015
BOOT_CLOJURE_VERSION=1.6.0
BOOT_VERSION=2.4.2
#App version: 2.0.0

jaen10:11:46

And the downloading thing keeps repeating on each invocation

juhoteperi10:11:50

Hmm the app version looks incorrect

juhoteperi10:11:41

I suggest removing or renaming ~/.boot/cache

jaen10:11:39

Hmm, I've just notice I have a ~/.boot/boot.properties which pointed to 2.1.2. I've cleared the cache subdir, the lib subdir and put 2.4.2 in the aforementioned file and it stopped

jaen11:11:05

Hm, is there something to set the env values that behaves like with-bindings so I can have the change scoped to single task only?

micha13:11:49

@jaen: unlike thread-local bindings, not all changes to the JVM can be "popped" back off the stack

micha13:11:17

like if you add dependencies to the classpath, there is no way to remove those jars from the classloaders they were loaded into

jaen13:11:43

Right. What what I had in mind was :resource-paths - boot-immutant plugin seems to require sources in the resource paths, but that feels weird to put them there, so I wanted to scope it only to the task that requires it.

micha13:11:38

there is a with-env macro floating around somewhere

micha13:11:45

i can't find it at the moment

juhoteperi14:11:10

@jaen: Often it's the correct way to add Clj sources to resource-paths

juhoteperi14:11:45

@jaen: Boot terms source-paths and resource-paths are a bit different from Leiningen's terms. Source-paths means files that should be present in classpath but not present in the JAR (sources that are compiled, like cljs->js, java->class, AOT compiled clj->class) and resource-paths are files which should be present JAR like non-AOT clj

jaen14:11:30

Ah, I see

dave18:11:45

hope someone finds it useful!

dave18:11:12

you can create standalone *nix executables like boot aot pom uber jar bin --output-dir bin, and windows exe files like boot aot pom uber jar exe --output-dir bin (though the exe task requires launch4j and a little bit of additional configuration via task options)

dave18:11:37

it will also work with existing jar files, i.e. boot jar --file target/my-project-0.1.0.jar --output-dir bin, same for the exe task

micha18:11:51

@dave :thumbsup: 🚀

micha19:11:28

Any help with testing 2.5.0-SNAPSHOT would be greatly appreciated

juhoteperi19:11:37

@micha: I guess we should work out the push options before the next release

micha19:11:49

yeah totally

juhoteperi19:11:20

One solution would be to replace repo-map option with repo-username, repo-password, repo-passphrase

juhoteperi19:11:26

Not sure if other options make sense?

juhoteperi19:11:18

Or should be encourage people to set up separate deploy repositories (like bootlaces does) and not provide those options for push task?

juhoteperi19:11:45

Boot could even provide default deploy-clojars repository with sensible username and password default options

juhoteperi19:11:32

Btw. the changelog for 2.5.0 looks really good!

juhoteperi19:11:38

Fixes many problems I have encountered

micha19:11:09

i think we want some kind of merging behavior, perhaps

micha19:11:20

i like the repo map i think

juhoteperi19:11:21

There is merging behaviour currently

micha19:11:39

does it rely on the repo names, or urls?

micha19:11:02

maybe we should merge by url

micha19:11:06

or url regex

micha19:11:24

that would allow the project to define repos

micha19:11:32

and your local creds could apply to them

juhoteperi19:11:42

It makes sense to select the used repo by name in push task

micha19:11:04

but when you have your creds file you want to associate password and stuff with the repo url i think

micha19:11:36

or at least use your own names there

micha19:11:42

maybe that's the best way?

micha19:11:01

then you could do the BOOT_REPO_USERNAME env var thing

juhoteperi19:11:04

Perhaps push task could use the similar logic to resolve-deps

micha19:11:36

how about this

micha19:11:28

we have a repositories.edn file, which the user uses to configure names and repo info

micha19:11:45

we can then have other files that merge on top of that

micha19:11:02

the repositories,edn file is to associate names with repo configs

micha19:11:38

so you'd have {:clojars {:url "htps://...."}}

micha19:11:46

in repositories.edn, at minimum

juhoteperi19:11:53

That's the same data that's on boot env?

juhoteperi19:11:10

What's the benefit of moving it from env to edn file?

micha19:11:14

then you could have credentials.edn.asc or whatever

micha19:11:30

the credentials file would be merged on top of the repositories file

micha19:11:40

env vars merged on that

micha19:11:44

and system props

juhoteperi19:11:52

How would that work with clojars where the creds should only be merged for deploy?

micha19:11:58

so when all the merging is done we have a database of repo configs

micha19:11:06

which we then index by url

micha19:11:28

so for each repo url we have info such as the name the user gave for it

micha19:11:37

the username, password etc

micha19:11:00

now suppose a project provides :repositories in via set-env!

micha19:11:26

suppose those are named like "snapshot" and "release"

micha19:11:29

like this

micha19:11:20

:repositories [["snapshot" {:url ""}]["releases" {:url ""}]]

micha19:11:34

when you do boot push -r snapshot

micha19:11:43

it would find the repo url in the env

micha19:11:56

and then it would be able to look that up in the index from the merged local stuff

micha19:11:03

and merge that on top of it

micha19:11:11

so you'd get usernames and passwords and whatever

micha19:11:26

and your own names wouldn't need to coincide with the ones in the project build.boot

juhoteperi19:11:01

The current creds file matches to repos by url regex

juhoteperi19:11:13

So it would be possible to implement this without much changes

micha19:11:33

it seems like we don't need to change anything?

juhoteperi19:11:45

But the problem is that it must be defined somewhere if the repository needs credentials for download and/or deploy

juhoteperi19:11:57

clojars doesn't need credentials for downloading but needs creds for deploymed

juhoteperi19:11:32

datomic repo needs creds for downloading

micha19:11:48

does clojars fail if you give creds for downloading?

juhoteperi19:11:29

Leiningen has separate repostory configurations for these. :repositories {"clojars" {:url "..."}}, :deploy-repositories {"clojars" {:url "..." :username :gpg :password :gpg}}

micha19:11:44

yeah i'm not sure that's a good idea

micha19:11:02

seems like the lazy solution

micha19:11:27

that's something you have complete control over via set-env!

micha19:11:41

like if you want to add some repos only for deploying you can do that in your deploy task

juhoteperi19:11:01

Yeah well as I mentioned, one solution would be to encourage using separate repositories for deploy repos

juhoteperi19:11:09

like deploy-clojars set up by bootlaces

micha20:11:37

ok, so we currently have --repo-map for the push task

micha20:11:07

which does what the :deploy-repositories thing does

micha20:11:13

so that's good

micha20:11:22

i would say that problem is solved

micha20:11:02

the problem we have now is that the creds are by url, and you can't have different creds for pushing and pulling?

juhoteperi20:11:28

Also, there's a small problem though, repo-map has to be set up in the project but different users can have different preferences to where to retrieve the creds

micha20:11:53

i think that's correct behavior

micha20:11:05

you'd set up repo-map in your build.boot file

micha20:11:25

that ensures that the jars get pushed to the right place

juhoteperi20:11:42

Maybe one user wants to use creds from the file and other from env variables

juhoteperi20:11:56

But maybe we don't need to support this

juhoteperi20:11:15

Afaik leiningen doesn't support different creds for pushing and pulling

micha20:11:27

i think we should maybe even remove the :env :gpg etc stuff

juhoteperi20:11:27

And I don't remember seeing any issues about this

micha20:11:37

we should just check those places

micha20:11:44

and use the first that is found

micha20:11:55

i.e. system prop, env var, creds file

micha20:11:59

in that order

juhoteperi20:11:16

Currently those options are used to define when to use any creds

micha20:11:39

i can't imagine that using valid creds would cause something to fail

micha20:11:47

that would be pretty wacky

micha20:11:03

if you have valid creds it should be fine to use them

juhoteperi20:11:21

One side-effect would be unnecessary gpg invocation any time dependencies are loaded

juhoteperi20:11:43

Though it should be memoized so it's ran only once per boot process

micha20:11:00

yeah it would pop up the thing and you'd have to type in your passphrase

juhoteperi20:11:15

Yeah depending on your gpg conf

micha20:11:40

gpg-agent has a pretty short cache time by default, on ubuntu

juhoteperi20:11:36

I'm using gnome-keyring as ssh/gpg agent, it saves the password on keyring which is opened when I login

juhoteperi20:11:02

Though I would rather throw last part of gnome away and use something simpler, but gpg-agent is too horrible to use

micha20:11:18

lol i got rid of gnome completely

micha20:11:32

we don't want to support gpg agent though

micha20:11:48

i mean people needing to get that correctly configured to be able to use boot

juhoteperi20:11:50

We don't have to, as gpg binary does that

micha20:11:03

yeah but there are many ways it can be broken

micha20:11:12

i know, i've been there lol

juhoteperi20:11:29

Yeah, OS X setup was even worse than Ubuntu configuration

micha20:11:44

yeah so i think we can't assume that gpg-agent works

micha20:11:03

so i guess we do need to make something more complicated

juhoteperi20:11:12

The password entry GUI is horrible by default, "I won't let you access other programs or paste any text here because that would be insecure" :<

micha20:11:14

with some kind of push/pull specific configs

micha20:11:41

i brb one sec

juhoteperi20:11:52

Looks like clojars works if creds are used for downloading deps

juhoteperi20:11:34

I think the current solution is as good as or perhaps a bit better fit than Leiningen's (separate :deploy-repositories) so if can't come up with better solution we can release it

juhoteperi20:11:55

One could be to introduce some special keys to repository maps, like ["clojars" {:url "..." :deploy {:username :gpg, :password :gpg}}]] where :deploy value would be merged with toplevel when doing deploy and dropped when downloading deps

juhoteperi20:11:10

Could be acceptable as the repository maps aren't even now completely identical to aether presentation, because of the :gpg and :env special values

micha20:11:08

how about

micha20:11:40

{"" {:authenticate [:pull] :username ... :password ...}
 ...

micha20:11:27

actually what if we eval the credentials file

micha20:11:32

instead of read

micha20:11:41

this way you could build it however you need to

juhoteperi20:11:56

That requires gpg invocation, which we probably want to avoid for cases where it's not necessary

micha20:11:23

that means we want to separate the secret from the non secret

micha20:11:48

which means we could add BOOT_HOME/repositories.edn

micha20:11:16

and BOOT_HOME/credentials.edn.asc would be merged on top of that

micha20:11:26

that makes total sense to me

juhoteperi20:11:12

What would the first one contain?

micha20:11:25

it would contain the info about repos that isn't secret, liket his

juhoteperi20:11:27

maps with url regex and options about where to get the creds from?

micha20:11:09

{"" {:authenticate [:push]} ...}

micha20:11:22

and other repo config that isn't secret

juhoteperi20:11:45

Perhaps URL and such should still be set with set-env!

juhoteperi20:11:53

and the file only used for setting cred source

micha20:11:11

yeah i think we would want to merge

juhoteperi20:11:22

I'm not sure

micha20:11:22

the repositories.edn file would provide defaults

micha20:11:37

the project env and --repo-map would override that

micha20:11:03

and if :authentication is needed then we'd merge in creds by the various ways

micha20:11:12

creds file, env var, system prop

micha20:11:21

in reverse order lol

juhoteperi20:11:46

Or perhaps the repo defaults are read from the env, and then repositories.edn could overwrite or add values to those and it could use the same regex matching as creds file currently

juhoteperi20:11:15

Could also make sense to allow setting the same :authentication option or such from set-env

micha20:11:24

yes, i think it should use the same regex matching

micha20:11:37

but i think that per project things should always override global things

micha20:11:51

so i would say the repos.edn file should be just the defaults

micha20:11:03

and the project could override that in the set-env! or --repo-map

juhoteperi20:11:24

What about boot default repositories?

micha20:11:42

yeah we should write those out to the repos file

micha20:11:51

on first boot

juhoteperi20:11:57

And if the repos.edn is the default, it can't use regex matching as there is nothing to match against

micha20:11:07

sure it can

micha20:11:21

it matches against the concrete ones in the project env or the defaults

micha20:11:52

we'd get the set of active repos from the env or boot default

micha20:11:00

then we'd find matching repos.edn entries

micha20:11:15

merge the repos from the env on top of the ones found in repos.edn then

micha20:11:23

i think that would be fine, no?

juhoteperi20:11:55

Ah, yes I think so

micha20:11:00

i was thinking that repos.edn wouldn't add anything to your project

juhoteperi20:11:10

Yes makes sense

micha20:11:15

it's just a database of metadata and config for repos that might be pulled in

juhoteperi20:11:05

the env sets the available repos and stuff from repos.edn is merged into that, but values from env taking precedence

micha20:11:10

we could actually make it a program

micha20:11:52

eh, i guess no need

micha20:11:59

we could eval it at least, though

micha20:11:10

it wouldn't really change anything in the case where it's a static map

micha20:11:25

but you could put logic in there to figure things out if you needed to

micha20:11:58

like if you wanted to implement some other env var naming convention of your own

micha20:11:00

or something

micha20:11:58

2.5.0 is going to have some good stuff in it

micha21:11:58

so in the shower i realized something i think

micha21:11:19

perhaps we don't need all the special conventions of env vars and credentials files and all that

micha21:11:25

or repos.edn

micha21:11:45

we could perhaps use profile.boot for this

micha21:11:21

(set! boot.pod/*configure-repositories*
  (fn [repo-map authenticate?]
    (-> {"" {:name "clojars" :username (if authenticate? (get-creds ...))}}
    (merge repo-map))))

micha21:11:38

and tasks would call that function, perhaps?

micha21:11:59

we could of course make a function that gets creds etc