Clojurians Log v2
cljdoc 2023-10-05
Hey folks, I'm trying to debug https://cljdoc.org/builds/72110, but could use some pointers as I'm newer to cljdoc and Clojure broadly.
• If I'm reading the stack trace right, https://github.com/cljdoc/cljdoc-analyzer/blob/v1.0.779/modules/metagetta/src/cljdoc_analyzer/metagetta/clojure.clj#L93 is resulting in the evaluation of https://github.com/protojure/lib/blob/v2.8.0/modules/core/src/protojure/protobuf/any.cljc#L8, whose ns form at the top spurs a search for com.google.protobuf on the classpath. The search fails, and in turn the cljdoc build fails.
• Before calling into the metagetta, among the jars the analyzer downloads is com/google/protobuf/protobuf-java/3.24.3/protobuf-java-3.24.3.jar from central, https://app.circleci.com/pipelines/github/cljdoc/builder/45815/workflows/3f71acdc-0da1-4731-8f49-bc5bde934ca5/jobs/62190?invite=true#step-105-3223_91. Direct link to jar https://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.24.3/protobuf-java-3.24.3.jar.
• It's true that the jar doesn't literally contain any of the files named com/google/protobuf__init.class, com/google/protobuf.clj or com/google/protobuf.cljc on classpath. Is this the core issue, in which case any library depending on com.google.protobuf/protobuf-java should have broken cljdoc build? And how might I resolve it, if so?
Hi @spencer794! I'll take a peek shortly!
Nice description BTW, thanks, easy to follow! Ok, if I just try this outside of cljdoc... I can reproduce your error:
$ clojure -Sdeps '{:deps {io.github.protojure/core {:mvn/version "2.8.0"}}}'
Downloading: io/github/protojure/core/2.8.0/core-2.8.0.pom from clojars
Downloading: io/github/protojure/io/2.8.0/io-2.8.0.pom from clojars
Downloading: io/github/protojure/io/2.8.0/io-2.8.0.jar from clojars
Downloading: io/github/protojure/core/2.8.0/core-2.8.0.jar from clojars
Clojure 1.11.1
user=> (require '[protojure.protobuf.any :as any])
Execution error (FileNotFoundException) at protojure.protobuf.any/eval142$loading (any.cljc:4).
Could not locate com/google/protobuf__init.class, com/google/protobuf.clj or com/google/protobuf.cljc on classpath.com.google.protobuf a Java package? Did you mean to :import it instead of :requireing it?Also is this a Clojure and ClojureScript library? By default, the cljdoc analyzer will also attempt ClojureScript analysis when it sees those .cljc files. Just a heads up. Your failure is for analysis under Clojure, but after you get that working, ClojureScript analysis will be next.
Am happy to clarify and help more if you need it.
isn't com.google.protobuf a Java package?Ooh, good catch, it is. Interestingly though, this code is trying to use it as though it were a Clojure file that defines functions like with functions like new-Any. There's no such Clojure file committed in this repo, but such a Clojure file would be created if the protojure Protobuf compiler were run on a file like https://github.com/protocolbuffers/protobuf/blob/v24.4/src/google/protobuf/any.proto.
I also notice that the only place the problematic file is being require'd is from https://github.com/protojure/lib/blob/2f2dc2e1d965f6862cf6c428d50e173651f0f4ba/test/test/protojure/protobuf_test.clj#L13 for the library. So perhaps the author's expectation is: If you're running tests, then you might have some compiled proto code higher on your classpath that would match com.google.protobuf than that Google-authored Java package of the same name. If there's a way to either a) tell cljdoc to ignore this file, or b) tweak things so cljdoc never discovers it, that might be appropriate if the file is only used for tests.
> is this a Clojure and ClojureScript library?
I'm actually not sure, but I'd guess it targets Clojure only. The companion protoc plugin emits files with cljc extension, but those files don't contain any reader macros so I'm unsure why the cljc extension was chosen.AFAICT, the entire directory https://github.com/protojure/lib/tree/v2.8.0/modules/core/src/protojure/protobuf is only used from tests, so excluding it from documentation would make sense (I think?).
I see that that test-only directory is currently included in the JARs that get distributed. After reading https://github.com/cljdoc/cljdoc/blob/master/doc/userguide/for-library-authors.adoc#getting-dependencies-right, I bet if I moved that directory to the tests directory it'd be omitted from the JAR, and in turn not visible to cljdoc's analyzer.
Oh, I assumed you were the author of this lib, @spencer794. Not the case?
Nope! Merely a user
Ah!
Yes, if a namespace is not part of a public API, adding :no-doc metadata to the namespace will exclude it from being explicitly loaded for cljdoc analysis.
Hm, could this particular failure be avoided by adding a :no-doc? I'd have expected that all the files (or at least the ns forms conventionally located at the top) still get evaluated (though perhaps not analyzed), and it's at eval-time that the FileNotFoundException is killing the analysis.
Regardless, I think given there's a whole directory full of files in this library's JAR that's test-only. I'll suggest moving them to another location in the repo in order to exclude them from the JAR.
Thank you so much for your help!
Yeah the :no-doc might do the trick if the failing namespace is not loaded by any other source namespace.
But... I expect after Clojure analysis passes, the next failure might be ClojureScript analysis. You https://github.com/cljdoc/cljdoc/blob/master/doc/userguide/for-library-authors.adoc#api-languages.