Fork me on GitHub
#clojars
<
2016-07-18
>
cap10morgan16:07:05

Anyone else seen this error trying to deploy a new release: Could not transfer metadata my-tmpl:lein-template/maven-metadata.xml from/to releases (): Read timed out? Tried 2 different releases and gotten this error both times. status says clojars is 100% up.

tcrawley16:07:57

@cap10morgan: yes, that's been reported, but we have no idea of the cause right now https://github.com/clojars/clojars-web/issues/546

tcrawley16:07:25

did the deploy seem to take a while, or did the timeout occur immediately?

cap10morgan16:07:33

ah, OK. pretty quickly.

tcrawley16:07:56

was the timeout on maven-metadata.xml both times?

tcrawley16:07:31

we are using maven-medata.xml as a trigger to validate the deploy, copy it from a tmp repo to the live repo (part of the atomic deploy feature), and, just recently, to copy the project to Rackspace cloudfiles in anticipation of serving the repo from a CDN

tcrawley16:07:39

it's possible that that is all taking too long

tcrawley16:07:51

depending on the default timeout in aether

daveliepmann16:07:18

@cap10morgan: I saw that error, but the release seemed to deploy successfully anyway.

tcrawley16:07:23

in the reported cases, the deploy actually succeeded, yeah

daveliepmann16:07:38

@tcrawley: the error seemed to occur within a few seconds

cap10morgan16:07:39

OK, that’s good at least 🙂

tcrawley16:07:41

@daveliepmann: was your error with maven-metadata.xml as well?

daveliepmann16:07:02

Yes: Could not transfer metadata uruk:uruk/maven-metadata.xml from/to clojars (https://clojars.org/repo/): Read timed out Failed to deploy metadata: Could not transfer metadata uruk:uruk/maven-metadata.xml from/to clojars (https://clojars.org/repo/): Read timed out

tcrawley16:07:11

good to know, thanks

cap10morgan16:07:39

@tcrawley: want me to do another one w/ some debugging output turned on?

tcrawley16:07:40

I suspect we may need to move uploading to cloudfiles to a queue that gets processed async

tcrawley16:07:26

@cap10morgan: let me add some timing output to the server side, give me a few minutes

cap10morgan16:07:32

OK, sounds good

tcrawley16:07:43

how big is the jar for the project you are deploying?

tcrawley17:07:19

those are pretty small, I was thinking it was large uploads to cloudfiles causing the issue

tcrawley17:07:18

@cap10morgan: @daveliepmann: I added some profiling to the validation and upload-to-cloudfiles code, and deployed a small test app

tcrawley17:07:44

It didn't fail, but did take almost 8 seconds to validate + upload

tcrawley17:07:56

950ms just to upload the 12k jar

tcrawley17:07:59

to cloudfiles

tcrawley17:07:14

so, if the timeout is 10s, we could very easily hit it

tcrawley17:07:14

yeah, it looks like 10s may be the default - I just recreated the issue by increasing the jar size to 23k

tcrawley17:07:59

the validate+upload took 9800ms, and it's safe to assume we're spending another 200ms outside of that process to get us over 10s

tcrawley17:07:39

I'm going to disable the cloudfiles upload for now, since we're not yet using the artifacts uploaded there

tcrawley17:07:38

@cap10morgan: @daveliepmann: the cloudfiles uploads are now disabled in production. I just deployed my testapp, and the deploy time went from ~10s to 200ms. Please let me know if you continue to see the problem

cap10morgan17:07:20

@tcrawley: cool, will do. thanks!