Fork me on GitHub
#cljdoc
<
2020-02-12
>
rschmukler18:02:15

When I go to manually trigger a rebuild on https://cljdoc.org/d/teknql/systemic/0.0.1-SNAPSHOT I get an exception. Is there anything I can do to rebuild?

Jimmy Miller18:02:59

There is a hidden rebuild button next to where it says scm info missing. I made it black in this screenshot but you should just barely be able to see it on hover.

rschmukler18:02:24

Sorry! I found the button okay, but when I click it the page reports an exception

rschmukler18:02:39

(No stack trace or anything is leaked, however)

Jimmy Miller18:02:52

(I know nothing just knew about that button) Did you look at the section of the FAQ about scm issues? https://github.com/cljdoc/cljdoc/blob/master/doc/userguide/faq.md#how-do-i-set-scm-info-for-my-project

rschmukler18:02:49

I think I got the SCM working - but to test it I need a rebuild... But if I click rebuild, it seems to cause cljdoc to hit an exception and crash. Presumably it's being logged somewhere

Jimmy Miller18:02:26

You'd need to push a new version for it to pick up that information

rschmukler18:02:57

For it to pick up new SCM info?

Jimmy Miller18:02:03

Maybe not a new version since it is a snapshot. But I really have no idea. Maybe someone more knowledgeable will have to reply.

rschmukler18:02:16

Cool cool. Thanks for the help anyway 🙂

martinklepsch13:02:27

It seems that the code that determines pom and jar URLs doesn’t work. Specifically find-artifact-repository doesn’t properly identify Clojars as the repo. It seems that something changed in Clojars’ CDN so that the URLs changed from: https://repo.clojars.org/teknql/systemic/0.0.1-SNAPSHOT/ to https://repo.clojars.org/index.html?prefix=teknql/systemic/0.0.1-SNAPSHOT/ We’re doing a HEAD request checking if the first URL returns 200 OK but it now returns 301 Moved Permanently .

martinklepsch13:02:42

Probably worth opening an issue on this if you don’t mind.

martinklepsch13:02:57

@U06SGCEHJ is the URL change intentional and do you happen to know if all Maven repos support the new ?prefix style URLs?

tcrawley14:02:31

Yes, I moved the backend storage from Rackspace Cloudfiles to S3 over the weekend. Cloudfiles supported "directory" browsing, but S3 does not. So I used a tool that allowed you to browse by redirecting listings to a top level index w/js that generates an index. I didn't realize that would break things.

tcrawley14:02:51

That's not something that other maven repos do as far as I know

tcrawley14:02:17

I can give it some more thought and see if there is an alternate way to restore directory browsing if needed

martinklepsch14:02:08

I’d be happy to special case Clojars if that’s whats needed but not quite seeing what specifically is necessary yet. Maybe curl -iL would be all that’s needed to detect if a thing exists.

martinklepsch14:02:49

But chances are there’s also other ways to do what cljdoc is doing that are a little bit more robust.

tcrawley14:02:28

What does clojars return if the path doesn't exist? If that still returns a 301 instead of a 404, I'd consider that a bug

martinklepsch14:02:58

This is the current code that determines if a project/version combination is available at a specific repository

martinklepsch14:02:42

It does return a 301 for projects/versions that don’t exist

martinklepsch14:02:11

eg. Does not exist:

curl -i 
Exists:
curl -i 

tcrawley14:02:10

Ok, I think I can at least return a 404 in that case. I won't be able to look at it or give it more thought until this evening (US-Eastern), unfortunately

martinklepsch14:02:43

It’s done when it’s done 🙂

martinklepsch14:02:51

Thanks for looking into it!

tcrawley03:02:01

Well, I've got some good news, some bad news, and some good news. Good news: I was able to figure out how to serve the directory index files from fastly w/o using a redirect to /index.html?prefix=foo/bar Bad news: we can either always return a 200 for a non-artifact path (`/foo/bar/` ), or always return a 404. That's because S3 will always return a 404 for a path, since it's really just a key prefix. I always return a 200 now, even for a prefix that doesn't exist. It will just be an empty index file. Good news: I put up a PR which I think resolves this issue for cljdoc: https://github.com/cljdoc/cljdoc/pull/378

martinklepsch15:02:20

Thanks very much @U06SGCEHJ, merged and deployed & it’s working 🙌

tcrawley15:02:03

My pleasure! I'm glad that worked.

tcrawley03:02:01

Well, I've got some good news, some bad news, and some good news. Good news: I was able to figure out how to serve the directory index files from fastly w/o using a redirect to /index.html?prefix=foo/bar Bad news: we can either always return a 200 for a non-artifact path (`/foo/bar/` ), or always return a 404. That's because S3 will always return a 404 for a path, since it's really just a key prefix. I always return a 200 now, even for a prefix that doesn't exist. It will just be an empty index file. Good news: I put up a PR which I think resolves this issue for cljdoc: https://github.com/cljdoc/cljdoc/pull/378