It could have something to do with the polylith dev.clj file. cljdoc basically scans your jar for .clj files and then tries to require each namespace it found. I’m not quite sure why it tries dev though instead of the full path based on it’s location in the jar.

If I moved the template .clj files to resources, would that solve the problem? These aren't valid Clojure files.

I think it would not since, as far as the jar is concerned, there’s no difference between files from src and files from resources

Good point. I guess I can rename them to not be .clj files since they sort of aren't really.

I think the irony here is that dev.clj is the only template .clj file that has legal syntax so, in the past, http://cljdoc.org has just ignored the .clj files because they are not syntactically legal...

Really? I’d think it tries to require every .clj file and fails if they’re invalid

It was happily importing the valid namespaces up until I added dev.clj -- that's the only file that is syntactically legal (but its ns does not match the path from src).

Anyway, I'm just renaming that one and I'll see whether it works 🙂

Yup, that worked: https://cljdoc.org/d/com.github.seancorfield/clj-new/1.1.317/doc/readme -- it ignores all the illegal syntax files, and I only needed to rename the one .clj file!

ClojureScript analysis might (not sure) offer some challenges here. If I remember correctly, an analyzed namespace will analyze namespaces required by that namespace. So reporting which namespace is an issue might require some care.

Also, if there are errors (or maybe we consider them warnings?) how will they be conveyed back to the library author? The library author might be totally ok with the warnings, and might feel they should not be conveyed to the person reading their docs.

If cljdoc could fail with good error messages, maybe a way of explicitly excluding nses (or maybe files) from analysis would be a more interesting problem to solve? That is, if unwanted nses being analyzed are a common cause of failure.

Personally, I’m ok with a lib that has invalid source code fails to be analyzed by cljdoc.

For me, that would be more helpful than it passing with warnings when my code is bad.

All just my opinions, of course.

Not a ton of thoughts given to that yet. Cljdoc ties everything to a release, which is something I really like. I guess we could use tags for that? In theory we should be able to work with source based libs since we interenally use tools.deps to build the classpath for analysis

I wonder how folks would submit a git repo project to cljdoc. Right now everything comes from clojars, which offers a sane barrier of entry. Would cljdoc need user accounts?

@lee I was thinking that for com.github.<username>/<repo> (and the io.* variant) you could deduce the GH URL the same way t.d.a does in the latest CLI releases. There's no reason to make a JAR and release it to Clojars for a lot of libs and tools that are specific to the CLI / deps.edn world now...

Yeah I do see your point. Makes sense. I was just wondering aloud how cljdoc would learn about JAR-less releases.

I'd assume it'd work basically the same way it does now. You just go to a link like https://cljdoc.org/d/io.github.igjoshua/farolero/1.2.0/ and then if there's nothing there it just derives the github repo and asks if you want to build it.

It'd derive the url from the groupname and artifact name pair, and then the tag would be in place of the version

@U5NCUG8NR I think that would technically work, yes. But I was wondering what might happen with this in the wild old Internet. I was trying to get across that clojars requires accounts and authentication and cljdoc only reads from clojars. Are we OK with any anonymous human/bot requesting that cljdoc build docs from a git repo? Will that be prone to abuse? Just something worth thinking about.

That's a reasonable concern, true. What sort of API/interaction does http://cljdoc.org have with http://clojars.org? Could you, for example, establish with a group ID (e.g., com.github.seancorfield) exists as a VGN on Clojars, and only accept GH source build requests for VGNs that Clojars knows about?

I'll observe that neither tools.build nor tools.tools exists on Maven Central so http://cljdoc.org cannot build those -- but would have to figure out that org.clojure was somehow valid (it's a group ID on Maven Central -- but not on Clojars) so VGNs are the only possible valid option for source doc builds...

Tagental but related thought: Anybody can request that cljdoc build public docs from any jar on clojars. That seems ok to me. But are there cases where a library author might not like it if someone builds docs from their git repo? Might be ok too, but something to think about.

Something to consider would be validating that a doc/cljdoc.edn file exists in the repo before attempting to clone it to ensure that the developer is opting in

That could both help prevent abuse and prevent repos from being added without developer consent. The only remaining problem that I see at the end of this is one that already exists, that is to require namespaces the cljdoc server is doing arbitrary code execution from random artifacts on maven or from github. That may present a huge security risk if not properly sandboxed.

Although I'm sure you've already taken that into account.

Yup, the security-risky bit is analyzing the API. Cljdoc respects the fact that some APIs are generated/modified at load time. This API eval work is safely sandboxed in a job at CircleCI.

> Something to consider would be validating that a doc/cljdoc.edn file exists in the repo before attempting to clone it to ensure that the developer is opting in That seems like a sensible approach to me.

Is there any conflict we need to think about between docs created for lib x at version y from clojars vs git repo. If the version tag is properly placed in the git repo, I suppose not?

Some interesting thoughts/discussion here for sure, I can create a git issue to try to capture sometime today.

The "nice" thing about being able to pull a tag-based release from GH is that a maintainer could tag a release, import it to cljdoc, make some doc changes, move the tag, re-run the cljdoc job and have the doc changes incorporated... Unless you want to go to the trouble of recording the SHA when a tag-based release is imported, and not letting it re-import if the SHA changes? That would restore the immutability of releases and also match the check t.d.a does on tag/sha pairs to some degree (o/c you can change the SHA in a coord if you want t.d.a to use a lib that updated its tag, so it's not really immutable there).

And I would say to check http://clojars.org first for a specific release and only if it is not present go to GH (to prevent folks "shadowing" actual JAR releases by moving tags on GH).

Here’s the git issue, corrections and additions are, as always, most welcome: https://github.com/cljdoc/cljdoc/issues/459