Hey 😊 I’m also not sure about this, some googling doesn’t give a clear answer either. Maybe let’s start with what you suggested and then we can see how it affects google indexing, adding more pages if it seems necessary.

Also maybe @U051KLSJF has more knowledge on this since he opened a few SEO related issue

Yeah I think just doing the current routes is enough

Ie the routes for the current pages

That would certainly be a good start, and get you a long way

The Googlebot can probably crawl from there

This presents a decent argument for having a /current

Because you get a limited crawl budget with Google, especially for a new site like cljdoc, it may take some time for pages to get re-indexed, so the search results for pages may be out of date

I noticed this today on a cljdoc search

Interesting.

@U7PQH43PS so what Daniel is saying is that instead of putting /d/group/artifact in the Sitemap we should put in /d/group/artifact/CURRENT so the indexing budget is spent on the most recent versions at first... Maybe we should make the link that gets users to the list of versions rel=nofollow so crawlers don’t get to those pages

Ah and @U051KLSJF, you mean /current as in a proper url segment not just for redirecting, right? :thinking_face:

Yeah

Will explain more tomorrow

@U050TNB9F Sounds good to me but I don’t get the last part about rel=nofollow

(cljdoc.config/db (cljdoc.config/config))

It’s a bit long I usually just put it in a def

Thanks! Yeah that’s a good idea.

I created a new storage api all-distinct-docs to retrieve all the ingested documentations. However, I don’t quite understand how to use the API. When I called (cljdoc.storage.api/all-distinct-docs db-spec)

CompilerException java.lang.IllegalArgumentException: No implementation of method: :all-distinct-docs of protocol: #'cljdoc.storage.api/IStorage found for class: clojure.lang.PersistentArrayMap, compiling:(/Users/dawran6/Developer/cljdoc/src/cljdoc/server/sitemap.clj:22:1)

It seems like I’m passing the wrong db-spec type into my API, right?

Oh I got this after typing out my question. I need to use ->SQLiteStorage to convert the type

Oops I broke something. CI build didn’t pass.

If it’s only relevant for tests maybe not including it in the jar would be an option as well?

good call 🙂