Very cool! How would you compare the goal of this to those of http://cljdoc.org? cljdoc has some limitations (requirements) like that the artifact needs to come out of a Maven repo, is that one perhaps?

The idea was more the social aspect of clojuredocs, where you can post your own example, notes and see alsos, I don't know if you can do this with cljdoc. And for extraction it uses clojure.tools.deps for downloading the libraries and uses clj-kondo for extracting the analysis. For now I don't parse the markdowns of the documentations, I just use kondo's data to create the data

Gotcha, that makes sense, thank you!

There's no user-contributed notes on cljdoc indeed

You can use tools.deps for the libs, but there is a gotcha, you need to use git libs for declaring the lib, because is a way that I found to generate source code links

interesting. I think in cljdoc it matches the namespaces to files in the git repo and creates the source links that way

It uses some heuristics that aren't 100% guaranteed to be correct but so far I think it worked for 99.9% of cases

I found the day that I had to create this requirement haha https://clojurians.slack.com/archives/CHY97NXE2/p1679674195357359

This is really neat! I have a project called dewey that runs clj-kondo analysis on all the clj projects I can find on github once a week, https://github.com/phronmophobic/dewey/releases. I was recently thinking about adding another step in the pipeline that feeds the analysis into datalevin and includes that as part of the weekly releases. I'm much better at making data pipelines than creating frontend webapps: • https://cloogle.phronemophobic.com/doc-search.html • https://phronmophobic.github.io/dewey/search.html Might be interesting to combine forces if you're interested.

We've considered clj-kondo for cljdoc a few times but sticked to tools.analyzer because it will also correctly capture vars defined by macros

@U7RJTCH6J I used dewey to select the first libs for the site yesterday @UL618PRQ9 showed me when I asked for help to find the most stared clojure libs, it's super useful

Yeah, I want to review all the data pipeline, it's the thing I didn't thought much, I would be super keen to entirely rewrite or use another source, do you have a way to identify that a repo is a library in someway?

> do you have a way to identify that a repo is a library in someway? A library as opposed to an application?

I don't do any filtering. I just include it if it's a clojure repository.

Even for most applications, most clojure namespaces are generally reusable so I'm not sure excluding whole repositories makes sense.

It might be interesting to be able to sort functions using some combination of relevance and usage.

Like if a var is used outside its namespace or project more, it's maybe more likely to be relevant.

That's the kind of stuff that might be interesting to include either in a data dump or datalevin dump.

Every var could have a precomputed :stats/num-usages attribute (or similar).

Makes total sense, I think kondo has something in the analysis to find about the function usage, but would be limited to the project itself

Right, but you can combine analyses to derive stats across the whole ecosystem.

This visualization is for namespace requires, but you could do the same for var-usages.

Incredible

wow that is very cool

I really like that.

It’s so great that people are working on this area! One of the things I miss most about the Haskell ecosystem is Hoogle. Not only could you search for functions in the standard library by name or type signature, but it was also very easy to generate your own hoogle site based on your build. Pretty much every company I’ve heard of using Haskell just internally publishes a custom Hoogle based on all of their dependencies, and since it’s based on their build, it works whether those deps are public or private. Adding community-generated docs just takes that to a whole new level!

@U050TNB9F > We've considered clj-kondo for cljdoc a few times but sticked to tools.analyzer because it will also correctly capture vars defined by macros This is true, but the only reason we don't also offer clj-kondo static analysis is that I did not finish the feature!

For the curious, here's a bit more on cljdoc API analysis: Cljdoc currently uses dynamic analysis to discover a library's public API. This means namespaces are actually loaded, so any parts of the API generated at load time are included. Because dynamic loading requires the namespaces to successfully load, it does not always succeed (sometimes there are optional dependencies, sometimes a library depends on a specific JDK or fails to load in some JDK versions, etc). Static analysis using clj-kondo has the advantage that it only examines the source code, doesn't load any namespaces, and only parses the source code. But as @U050TNB9F mentioned, it will miss dynamically generated vars (although it does understand and compensate for potemkin import-vars). While adding static analysis to cljdoc (a WIP), I started https://github.com/cljdoc/cljdoc/issues/543#issuecomment-1694509643, which might be of interest.