Fork me on GitHub
#boot
<
2016-01-02
>
danielsz01:01:57

Oh, boot -h is trying to connect to a server but INFO: I/O exception (java.net.SocketException) caught when connecting to the target host: Network is unreachable

danielsz01:01:59

It's gone now, but probably a good idea to make the failure silent if network can't be reached.

danielsz01:01:36

Oh, sorry, it seems that Clojars is down. I forgot about profile.boot

tolitius02:01:44

@flyboarder: offline does not give me a jar :) @seancorfield: 0.2.1 is better than no jar. thx

flyboarder02:01:10

@tolitius: i was asking if there was a flag, sorry for confusion

tolitius03:01:44

@flyboarder: all good, caused by the timing issue simple_smile

limist10:01:04

Help needed please: without having knowingly changed anything, when I try running boot, I'm now seeing endless repeats of this error message:

Jan 02, 2016 10:56:37 AM org.apache.http.impl.client.DefaultRequestDirector tryConnect

INFO: I/O exception (java.net.SocketException) caught when connecting to the target host: Network is unreachable
There's nothing wrong with my network, ping'ing or loading external websites etc. works fine. Any ideas how to get boot working again? Thanks!

zharinov10:01:11

this is DDoS on linode where clojars hosted :white_frowning_face:

limist10:01:10

@s_zharinov: Wow, a DDOS that's been running for almost a day?

limist10:01:18

If the DDOS is at cause, is there a way to tell Boot to temporarily avoid external network access?

zharinov10:01:43

i think no, unfortunately

limist10:01:05

Thanks for the explanation, I least I know it's not my machine/fault. simple_smile But damn, this external dependency sucks.

zharinov10:01:09

you can ping clojars periodically, it was available for couple minutes last hour

escherize10:01:27

Is there a build.boot skeleton for starging a {app, task, ...?}?

escherize10:01:02

I've just been copying around a template

escherize10:01:46

My previous default was lein new luminus <project>

limist10:01:50

@s_zharinov: Thanks for the idea, I just let the boot process run; it got through eventually.

danielsz14:01:21

That DDoS been running for the whole week.

micha17:01:47

you should be able to do (set-env! :offline? true), by the way

micha17:01:33

obviously this doesn't apply to boot's own internal bootstrapping process, like if you're running boot for the first time

tcrawley17:01:39

anyone want to help test a clojars mirror?

tcrawley17:01:05

for lein, you use

:mirrors {#"clojars" {:name "clojars mirror"
                             :url ""}}

micha17:01:33

ah yeah interesting

micha17:01:45

boot doesn't have that, but i see the value of it

micha17:01:08

i test one sec

tcrawley17:01:27

dammit, I can't figure out this slack thing :)

micha17:01:22

boot does understand about the :mirrors key

tcrawley17:01:23

I was about to get excited!

micha17:01:29

in set-env!

micha17:01:41

no, get excited, the mirror totally works!

tcrawley17:01:04

is there a way to set-env! globally?

micha17:01:24

you can makea ~/.boot/profile.boot file

micha17:01:31

with expressions in it

micha17:01:47

those will be evaluated before the build.boot expressions

micha17:01:32

poor linode

tcrawley17:01:26

@micha: cool - would you mind adding the relevant instructions to https://github.com/clojars/clojars-web/wiki/Mirrors?

micha17:01:29

still testing boot

micha17:01:49

but the mirror definitely works with leiningen for me

micha17:01:59

it's difficult to test becasue clojars keeps coming back online intermittently

tcrawley17:01:13

I should just kill it :)

micha17:01:31

you could move dns to the mirror?

tcrawley17:01:41

well, that would break deploys

tcrawley17:01:08

you could blackhole http://clojars.org in /etc/hosts for testing

alandipert17:01:51

i tried that and (set-env! :mirrors ... in my profile.boot doesn't seem to help 😦

alandipert17:01:38

hm, i can un-blackhole http://clojars.org by pointing to your mirror in etc/hosts

tcrawley17:01:59

that also works :)

micha17:01:15

will that corrupt repo metadata?

alandipert17:01:34

err, no dice - certs

tcrawley17:01:47

good question, I'm not sure

tcrawley17:01:32

with :mirrors, it probably treats the mirror as the same as the src for metadata purposes

tcrawley17:01:50

alandipert: what error are you seeing? and what OS + java version?

alandipert17:01:21

Caused by: org.sonatype.aether.resolution.ArtifactResolutionException: Could not transfer artifact alandipert:desiderata:pom:1.0.2 from/to clojars (): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

alandipert17:01:33

that's after i added 104.236.235.3 to my /etc/hosts

tcrawley17:01:34

ah, in that case, the hostname wouldn't match the cert. try replacing clojars in :repositories with the mirror url

micha17:01:14

i'm pretty sure that will cause metadata conflicts

micha17:01:28

well i'm 50% sure

alandipert17:01:39

it depends on what i already have locally right?

micha17:01:16

well the problem is that maven stores the nameo for the repo you downloaded from with the local jar

micha17:01:29

if you later have a project with clojars in the :repositories it will fail

micha17:01:39

because it sees the same coords but from a different repo

micha17:01:55

but it fails without telling you a good explanation of why

micha17:01:07

so you end up rm -rf ~/.m2 and it works again

alandipert17:01:19

i'm cool with that

micha17:01:29

the essence of the issue is that local m2 can only contain one instance of a given artifact

micha17:01:56

and in the java world people push artifacts with different contents to different repos but with the same coords

micha17:01:06

so maven will try to protect you from that

micha17:01:02

seems like a bad decision when they designed the maven stuff, not to have the artifact coordinates be just the url to the pom

micha17:01:46

if you want to have a poxy like nexus it would be easy enough to just have varnish or nginx handle those urls

tcrawley17:01:39

well, http://clojars.org is down again, so now's a good time to test mirrors :)

alandipert17:01:08

@micha: that's ivy's major plus, right?

alandipert17:01:13

iirc they have a url resolver thing

micha17:01:31

man, i still want to try using ivy in boot

micha17:01:26

:mirrors doesn't seem to work in boot for me

micha18:01:05

oh wait maybe i'm dumb

micha18:01:26

the only thing is it doesn't work for boot's own jars

micha18:01:35

like when boot is bootstrapping itself

micha18:01:06

i think maybe we need a data file that boot can use to get repository info

alandipert18:01:20

or a BOOT_REPO env var?

micha18:01:32

yeah something

micha18:01:14

i updated the wiki

alandipert18:01:44

@tcrawley: i'm gettin a cert error using lein w/ the mirror,

Could not transfer artifact clojure-complete:clojure-complete:pom:0.2.3 from/to clojars mirror (): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

tcrawley18:01:33

what version of java? justin_smith is seeing the same issue with an early java 8 build

alandipert18:01:48

oracle 1.8.0_66

tcrawley18:01:12

that's a newer build than I, but I'm on openjdk

tcrawley18:01:32

I suspect it's an issue with the cert chain for letsencrypt

alandipert18:01:50

i'll switch to openjdk and see how it do

tcrawley18:01:08

I may need to buy a cert for the mirror

tcrawley19:01:40

@alandipert: I replaced the cert - things should work betterer for you now

micha19:01:00

did you use letsencrypt for the new cert?

tcrawley19:01:40

no, I bought the new one from comodo

micha19:01:59

i was pretty excited about letsencrypt

micha19:01:15

i bet letsencrypt is run by the NSA

micha19:01:44

lol joking of course

micha19:01:38

@tcrawley: re: your recent PR, the gpg signature verification, do you consider gpg signatures on jars to be an adequate solution there?

micha19:01:03

i mean couldn't someone make a malicious jar and just sign it with any gpg key?

micha19:01:21

like if someone managed to get my clojars creds and uploaded a boot jar with malicious code in it, couldn't they just sign the jar with any disposable gpg key and upload the public key to the mit keyserver?

tcrawley19:01:38

letsencrpyt is pretty sweet, and will be handy in the future once keystores get updated

tcrawley19:01:16

you are correct re:signing - it is really only useful if you can verify the artifacts are signed by certs that are in your web of trust

tcrawley19:01:37

showing things are signed is just a security blanket

micha19:01:18

i wonder if maybe we should incorporate web of trust in the --verify-deps implementation?

micha19:01:58

i guess that was the TODO in there, the :trusted state

arohner19:01:03

hi, I’m trying to get started with boot, and I’m seeing

$ boot -h
Jan 02, 2016 1:51:39 PM org.apache.http.impl.client.DefaultRequestDirector tryConnect
INFO: I/O exception (java.net.NoRouteToHostException) caught when connecting to the target host: No route to host
. I assume it’s because of the clojars mess, is there a workaround?

arohner19:01:22

micha: I have that. I see 'Note: Boot does not currently support using mirrors for the initial downloading of its own dependencies (eg., the first time a new version of Boot is run).’. I guess this is my first time

arohner19:01:38

can I do something like install from source?

micha19:01:53

you can do that

micha19:01:15

just add the :mirrors to the project.clj files in the boot project

micha19:01:53

you could also use leiningen to fetch the boot jars

micha19:01:00

and install them to local maven

micha20:01:00

you need boot/boot boot/worker boot/aether boot/pod and boot/core

micha20:01:06

version 2.5.5

micha20:01:28

if you fetch those with leiningen you should be good to go

alandipert20:01:04

you also need to configure the mirror in maven

alandipert20:01:45

hm, maybe not - if you comment out 'base' at the end of the makefile

micha20:01:51

yeah fetching the jars seems like a better route than building

arohner20:01:53

I have :mirrors in my lein profile, I assume that would work? Just set the maven mirror, we’ll see

alandipert20:01:03

i think that should work... i'm trying it now myself

micha20:01:08

yeah, just make a project that depends on those deps

micha20:01:17

and do lein deps or whatever

micha20:01:21

i think maybe i'll update boot to look at the MIRRORS page in github clojars-web

micha20:01:28

and configure itself

alandipert20:01:32

(defproject foo "1"
  :dependencies [[boot/boot "2.5.5"]
                 [boot/worker "2.5.5"]
                 [boot/aether "2.5.5"]
                 [boot/pod "2.5.5"]
                 [boot/core "2.5.5"]])

alandipert20:01:31

@micha: into it but that should be another BOOT_* var probly?

alandipert20:01:24

if it were a var that would be helpful for running a private mirror

alandipert20:01:35

i suspect every hardcoded URL everywhere in boot should be an env var

micha20:01:57

yeah that's true

arohner20:01:34

looks like it’s still trying to connect to something:

arohner20:01:43

bahamut:boot arohner$ lein deps
Retrieving boot/boot/2.5.5/boot-2.5.5.pom from tcrawley
Retrieving boot/worker/2.5.5/worker-2.5.5.pom from tcrawley
Retrieving boot/core/2.5.5/core-2.5.5.pom from tcrawley
Retrieving boot/boot/2.5.5/boot-2.5.5.jar from tcrawley
Retrieving boot/worker/2.5.5/worker-2.5.5.jar from tcrawley
Retrieving boot/core/2.5.5/core-2.5.5.jar from tcrawley

micha20:01:46

what does boot -V say?

arohner20:01:01

bahamut:rasterize arohner$ boot -h
Jan 02, 2016 2:06:35 PM org.apache.http.impl.client.DefaultRequestDirector tryConnect
INFO: I/O exception (java.net.NoRouteToHostException) caught when connecting to the target host: No route to host

arohner20:01:13

ah, boot 2.4.2

micha20:01:28

do BOOT_VERSION=2.5.5 boot -h

micha20:01:51

you can set this in ~/.boot/boot.properties

arohner20:01:57

$ BOOT_VERSION=2.5.5 boot -h
Downloading .
Jan 02, 2016 2:10:02 PM org.apache.http.impl.client.DefaultRequestDirector tryConnect
INFO: I/O exception (java.net.NoRouteToHostException) caught when connecting to the target host: No route to host

micha20:01:48

try one more time

micha20:01:58

actually edit the boot.properties file

micha20:01:01

and try again

alandipert20:01:43

no joy for me

micha20:01:58

dunno ¯\(ツ)/¯

arohner20:01:53

same result after editing boot.properties

micha20:01:50

hm perhaps there is a solution via boot-shim.clj

alandipert20:01:08

i blackholed clojars in /etc/hosts and then ran boot -h

micha20:01:33

i don't get it

micha20:01:35

it works for me

alandipert20:01:37

i think in App it's trying to get BOOT_CLOJURE_VERSION from clojars

alandipert20:01:49

did you blow away m2?

micha20:01:10

well i blew away some things

micha20:01:28

yeah maybe it's clojure

micha20:01:48

if you don't have clojure in m2 that would have the same effect

alandipert20:01:26

ok i blew away m2, then i did lein deps in my 'seed' project, then i did boot -h and got this: https://gist.github.com/alandipert/a5c54b04ab8505a38f99

alandipert20:01:33

(clojars blackholed)

alandipert20:01:21

it appears now that it's trying and failing getting boot pod

micha20:01:51

which is already in your m2

micha20:01:54

presumably

micha20:01:29

that doesn't make sense to me

micha20:01:14

strace will show which files in local m2 it's looking for

micha20:01:32

before it tries to contact clojars

alandipert20:01:18

weird it doesn't appear to look at anything in m2

alandipert20:01:23

in App that is

mindbender20:01:18

How should I be using :mirror with boot? I already did merge-env! still getting the netwok unreachable thing.

micha20:01:52

@mindbender: you had boot working before the clojars outage?

mindbender20:01:34

I even have a running boot instance already and just trying to use cemerick.pomegranate to add dependencies.

mindbender20:01:34

I already checked boot-env for :mirror and the value is already there.

micha20:01:05

it's :mirrors, right?

micha20:01:11

not :mirror

mindbender20:01:56

Yes. it is. Just mispelled it here.

alandipert20:01:17

might it fail because in App we set update-always?

micha20:01:20

you're using pomegranate directly?

micha20:01:24

or via set-env!

alandipert20:01:25

that should only affect snapshots, right?

micha20:01:52

set-env! should work if the env has :mirrors configured

mindbender20:01:55

@micha: Yes. From the repl

mindbender20:01:19

It's a usual for me

mindbender20:01:20

I also have set-env! configured for my next boot up

mindbender20:01:41

Or rather merge-env!

micha20:01:01

i dunno, it's a puzzle

micha20:01:06

i can't reproduce here

mindbender20:01:28

@micha: is the :mirrors working for you?

micha20:01:32

adding -v option to boot will show debug info

micha20:01:46

yes, mirrors are working fine for me

micha20:01:03

the debug info contains a lot of stuff about maven operations

micha20:01:12

you can set it at the repl

micha20:01:26

(swap! boot.util/*verbosity* inc)

micha20:01:39

then try set-env! add deps

mindbender20:01:40

I only have tcrawley mirror in :mirrors

mindbender21:01:02

I only needed to add the :mirrors key to cemerick.pomegranate directly

mindbender21:01:45

Thanks @tcrawley for the mirror!

micha21:01:13

@mindbender: set-env! doesn't work?

alandipert21:01:20

@micha: i think the problem (for me at least) is https://github.com/boot-clj/boot/blob/master/boot/base/src/main/java/boot/App.java#L363 - it tries to talk to clojars before ever looking at disk, researching maven UPDATE_ALWAYS this looks to also apply to releases

micha21:01:38

we need to hoist that to a static property of the class

micha21:01:46

and only set it when we're updating

micha21:01:21

i have a patch one sec

alandipert21:01:46

i wonder also about deploying boot's jars to S3

micha21:01:07

we have the same problem really

micha21:01:30

it would probably be easier to manage though

alandipert21:01:01

i like the idea of the user coming up in an environment where anything they need to resolve they have control over where it comes from

alandipert21:01:14

i guess even w/ the S3 repo there should be an env var

alandipert21:01:17

BOOT_MAVEN_REPO

micha21:01:25

yeah i mean turtles

micha21:01:42

at some point you just download the source and mess with it there

alandipert21:01:21

but to build the source you need some deps

alandipert21:01:29

i guess you build them too

micha21:01:33

yeah you can build those too

micha21:01:47

i mean if clojars is down you're going to have a hard time in clojure

alandipert21:01:29

yeah but the more repos you can configure the easier it is to mitigate

micha21:01:36

i guess s3 is a better solution all around to the combination of github and clojars

micha21:01:01

but it would be a breaking change, or we'd have to continue to push things to all the places

micha21:01:09

and then you have maven issues

alandipert21:01:28

it's breaking and we haven't changed

micha21:01:54

well sort of

micha21:01:05

in the same way that smashing your computer witha rock is a breaking change

micha21:01:17

not really software related though

alandipert21:01:49

the problem is we have places embedded in code instead of coming from env

micha21:01:51

i agree that we need some way to at least set mirrors

micha21:01:56

i think maybe a default-repositories.properties file might be the thing

micha21:01:36

and boot could create that when it installs itself

micha21:01:50

default-repositories-2.5.5.properties

micha21:01:08

yeah i dunno

micha21:01:33

i think when all's said and done the boot-shim.clj way is going to be the simplest

alandipert21:01:32

how can it be? by the time shim is in play boot is already running

micha21:01:49

no, it's evaled before and dependencies are added

alandipert21:01:59

even before e.g. boot.pod is resolved?

micha21:01:39

in the case that you're installing boot for the first time, yes

micha21:01:52

because then boot will use its internal uberjar to resolve dependencies

micha21:01:57

which itself has no dependencies

micha21:01:06

boot-shim.clj will be run in there

micha21:01:15

which will affect how it downloads the rest of boot

alandipert21:01:35

all i'm advocating is an env var to inform the parts that download boot itself

alandipert21:01:41

after that we're in clojure programming land

micha21:01:05

so if you change the repo you load boot from, you will need to rm -rf m2 to use boot without that special configuration

micha21:01:21

because the local m2 will be poisoned

micha21:01:53

adding a mirror avoids that

micha21:01:56

is there a case where a mirror wouldn't work?

alandipert21:01:46

so you're saying env var for mirror instead of canonical ?

micha21:01:36

yeah so you can set a mirror for clojars and/or maven central when boot starts via env var

micha21:01:38

and boot could prepopulate it with info from the internet when it installs and when you do boot -u perhaps

micha21:01:09

there are security concerns with that that would need to be worked out

micha21:01:24

the automatic part

micha21:01:31

so maybe that's a bad idea

alandipert21:01:46

i would say there are no mirrors by default

alandipert21:01:16

is that still a problem?

micha21:01:44

no that's reasonable i think

alandipert22:01:02

@arohner: sorry, you picked a bad day to try boot 😦

alandipert22:01:29

we'll have to sort it out this week for work

arohner23:01:54

@alandipert: no problem. I was trying boot because I found several lein bugs related to the clojars mess

micha23:01:20

there we go: BOOT_CLOJARS_MIRROR and BOOT_MAVEN_CENTRAL_MIRROR

micha23:01:32

tested with blackhole and i think works

micha23:01:50

hard to simulate the issue though