Fork me on GitHub
#clojars
<
2022-12-28
>
genmeblog09:12:33

Hi. I observe some problems with deploying to Clojars now. Getting 500 Internal Server Error. Moreover, Clojars thinks that given version is deployed (more in a thread)

genmeblog09:12:10

$ lein deploy clojars
If there are a lot of uncached dependencies this might take a while ...
Created /home/tsl/clojure/clojisr/target/clojisr-1.0.0-BETA21.jar
Wrote /home/tsl/clojure/clojisr/pom.xml
Need to sign 2 files with GPG
[1/2] Signing /home/tsl/clojure/clojisr/target/clojisr-1.0.0-BETA21.jar with GPG
gpg: using "xxx" as default secret key for signing
[2/2] Signing /home/tsl/clojure/clojisr/pom.xml with GPG
gpg: using "xxx" as default secret key for signing
Sending scicloj/clojisr/1.0.0-BETA21/clojisr-1.0.0-BETA21.jar (233k)
    to 
Could not transfer artifact scicloj:clojisr:jar:1.0.0-BETA21 from/to clojars (): Transfer failed for  500 Internal Server Error
Sending scicloj/clojisr/1.0.0-BETA21/clojisr-1.0.0-BETA21.pom (15k)
    to 
Could not transfer artifact scicloj:clojisr:pom:1.0.0-BETA21 from/to clojars (): Transfer failed for  500 Internal Server Error
Sending scicloj/clojisr/1.0.0-BETA21/clojisr-1.0.0-BETA21.jar.asc (1k)
    to 
Could not transfer artifact scicloj:clojisr:jar.asc:1.0.0-BETA21 from/to clojars (): Authorization failed for  403 Forbidden
Sending scicloj/clojisr/1.0.0-BETA21/clojisr-1.0.0-BETA21.pom.asc (1k)
    to 
Could not transfer artifact scicloj:clojisr:pom.asc:1.0.0-BETA21 from/to clojars (): Authorization failed for  403 Forbidden
Failed to deploy artifacts: Could not transfer artifact scicloj:clojisr:jar:1.0.0-BETA21 from/to clojars (): Transfer failed for  500 Internal Server Error

genmeblog09:12:16

The same with a SNAPSHOT of other library:

genmeblog09:12:42

~/clojure/fastmath$ lein deploy clojars
Compiling 13 source files to /home/tsl/clojure/fastmath/target/classes
warning: [options] bootstrap class path not set in conjunction with -source 8
1 warning
Created /home/tsl/clojure/fastmath/target/fastmath-2.1.9-SNAPSHOT.jar
Wrote /home/tsl/clojure/fastmath/pom.xml
Retrieving generateme/fastmath/2.1.9-SNAPSHOT/maven-metadata.xml (1k)
    from 
Sending generateme/fastmath/2.1.9-SNAPSHOT/fastmath-2.1.9-20221228.095432-19.jar (238k)
    to 
Could not transfer artifact generateme:fastmath:jar:2.1.9-20221228.095432-19 from/to clojars (): Transfer failed for  500 Internal Server Error
Sending generateme/fastmath/2.1.9-SNAPSHOT/fastmath-2.1.9-20221228.095432-19.pom (5k)
    to 
Could not transfer artifact generateme:fastmath:pom:2.1.9-20221228.095432-19 from/to clojars (): Authorization failed for  403 Forbidden
Failed to deploy artifacts: Could not transfer artifact generateme:fastmath:jar:2.1.9-20221228.095432-19 from/to clojars (): Transfer failed for  500 Internal Server Error

genmeblog09:12:48

And when trying to use other library relying on a fastmath it tries to download the latest (failed to deploy) snapshot:

~/clojure/clojure2d$ lein repl
Could not find artifact generateme:fastmath:jar:2.1.9-20221228.095432-19 in clojars ()
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
Could not resolve dependencies

tcrawley15:12:43

This was caused by a misconfiguration in our terraform that caused the root volume to be too small, and it filled up.

👍 1
lread17:12:19

Hiya! I'm seeing what seems to be an invalid feed.clj.gz just now.

lread17:12:16

If I download like so:

❯ wget 
URL transformed to HTTPS due to an HSTS policy
--2022-12-28 12:21:58--  
Resolving  ()... 3.18.17.202, 18.217.66.205, 3.129.77.116, ...
Connecting to  ()|3.18.17.202|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location:  [following]
--2022-12-28 12:21:58--  
Resolving  ()... 151.101.1.128, 151.101.65.128, 151.101.129.128, ...
Connecting to  ()|151.101.1.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 139264 (136K) [application/gzip]
Saving to: 'feed.clj.gz'

feed.clj.gz                                               100%[====================================================================================================================================>] 136.00K  --.-KB/s    in 0.1s    

2022-12-28 12:21:59 (1.33 MB/s) - 'feed.clj.gz' saved [139264/139264] 
And then check like so, I see a problem:
❯ gzip -t feed.clj.gz                     

gzip: feed.clj.gz: unexpected end of file

tcrawley21:12:49

@UE21H2HHD Is this still an issue? It gets regenerated hourly, and the latest one works for me. The gz is 2.63MB and expands successfully to a 13MB file.

lread21:12:37

Ya, just tried again and am seeing the same thing... Am I doing something goof-ball-ish?

lread21:12:20

Ya... no matter how I download it, curl, wget, browser, I'm getting the wee 136kb file.

lread21:12:57

Any other lurkers wanna give downloading http://clojars.org/repo/feed.clj.gz a try?

lread21:12:43

Hmmm... just tried from my mac laptop and it seems to work, but not from my linux desktop.

lread21:12:01

dns cache maybe?

lread21:12:49

Weird, I'm on PopOS, so tried resolvectl flush-caches to clear DNS cache. But still same result. Rebooted. Same result. Seems somehow a my-side thing @U06SGCEHJ? But I'm not sure what's up yet.

lread22:12:57

If I do the wget with debug and verbose, I see:

❯ wget -d -v 
Setting --verbose (verbose) to 1
DEBUG output created by Wget 1.21.2 on linux-gnu.

Reading HSTS entries from /home/lee/.wget-hsts
URI encoding = 'UTF-8'
Converted file name 'feed.clj.gz' (UTF-8) -> 'feed.clj.gz' (UTF-8)
--2022-12-28 17:07:31--  
Resolving  ()... 18.217.66.205, 3.18.17.202, 3.129.77.116, ...
Caching  => 18.217.66.205 3.18.17.202 3.129.77.116 2600:1f16:90b:ac22:c6b:c6a9:a649:fb0b 2600:1f16:90b:ac00:a6de:5dfd:737b:7bcd 2600:1f16:90b:ac11:c746:a486:7826:f7ea
Connecting to  ()|18.217.66.205|:443... connected.
Created socket 3.
Releasing 0x0000559ff64bd470 (new refcount 1).
Initiating SSL handshake.
Handshake successful; connected socket 3 to SSL handle 0x0000559ff64bf210
certificate:
  subject: CN=
  issuer:  CN=Amazon RSA 2048 M02,O=Amazon,C=US
X509 certificate successfully verified and matches host 

---request begin---
GET /repo/feed.clj.gz HTTP/1.1
Host: 
User-Agent: Wget/1.21.2
Accept: */*
Accept-Encoding: identity
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 302 Moved Temporarily
Date: Wed, 28 Dec 2022 22:07:31 GMT
Content-Type: text/html
Content-Length: 154
Connection: keep-alive
Server: nginx
Location: 
Expires: Wed, 04 Jan 2023 22:07:31 GMT
Cache-Control: max-age=604800
Strict-Transport-Security: max-age=31536000

---response end---
302 Moved Temporarily
Registered socket 3 for persistent reuse.
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = false
Added new HSTS host:  (max-age: 31536000, includeSubdomains: false)
Location:  [following]
Skipping 154 bytes of body: [<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
] done.
URI content encoding = None
Converted file name 'feed.clj.gz' (UTF-8) -> 'feed.clj.gz' (UTF-8)
--2022-12-28 17:07:31--  
Resolving  ()... 151.101.1.128, 151.101.65.128, 151.101.129.128, ...
Caching  => 151.101.1.128 151.101.65.128 151.101.129.128 151.101.193.128
Connecting to  ()|151.101.1.128|:443... connected.
Created socket 4.
Releasing 0x0000559ff64f2140 (new refcount 1).
Initiating SSL handshake.
Handshake successful; connected socket 4 to SSL handle 0x0000559ff64cebc0
certificate:
  subject: CN=
  issuer:  CN=GlobalSign Atlas R3 DV TLS CA 2022 Q3,O=GlobalSign nv-sa,C=BE
X509 certificate successfully verified and matches host 

---request begin---
GET /feed.clj.gz HTTP/1.1
Host: 
User-Agent: Wget/1.21.2
Accept: */*
Accept-Encoding: identity
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 139264
x-amz-id-2: xDuywazpRGrbdB5Zx0Ww4RL+Dv0Il8q1jSZNN0H04b6lih636xohMVjcM9MO4DjQfKj7wNVuFPU=
x-amz-request-id: 5G1VAHW21X5Y7F2T
Last-Modified: Wed, 28 Dec 2022 15:00:32 GMT
ETag: "c5d51e731af57e0899099037aab65b6a"
Content-Type: application/gzip
Server: AmazonS3
Accept-Ranges: bytes
Date: Wed, 28 Dec 2022 22:07:32 GMT
Via: 1.1 varnish
Age: 22718
X-Served-By: cache-lga21935-LGA
X-Cache: HIT
X-Cache-Hits: 1
X-Timer: S1672265252.026943,VS0,VE11

---response end---
200 OK
Disabling further reuse of socket 3.
Closed 3/SSL 0x0000559ff64bf210
Registered socket 4 for persistent reuse.
Length: 139264 (136K) [application/gzip]
Saving to: 'feed.clj.gz.15'

feed.clj.gz.15                               100%[=============================================================================================>] 136.00K  --.-KB/s    in 0.08s   

2022-12-28 17:07:32 (1.69 MB/s) - 'feed.clj.gz.15' saved [139264/139264]

Saving HSTS entries to /home/lee/.wget-hsts

lread22:12:46

I see a different Etag from my mac laptop for the successful download from firefox: "07da6536d01ad252f8a7f1391b480e8c"

lread22:12:18

Feels like I'm hitting some bad version in some cache... but dunno. Will try again tomorrow and see if things have righted themselves.

tcrawley23:12:37

It is cached by Fastly, so you may be served by a node that has cached a bad copy. But there is a rule in Fastly to expire that file hourly, and fetch the original from s3. So I would expect this to resolve within an hour. I'll try doing a global purge of that URL.

tcrawley23:12:52

I've purged it.

lread04:12:17

Thanks a bunch, @U06SGCEHJ, I'm getting the proper full untruncated feed goodness now!

lread04:12:56

Something got stuck somewhere and it seems maybe your purge fixed it.

tcrawley11:12:17

My pleasure!