Fork me on GitHub
#clj-kondo
<
2022-12-01
>
sheluchin16:12:15

Hey @borkdude, I've been looking for ways to relate clj-kondo usage analysis data to source libraries rather than just source namespaces. I've kind of asked about this in a few different places, but ultimately it seems to come down to operating on :lib values. Do you have any thoughts on the feasibility of including :lib keys in the analysis output? https://clojure.org/reference/deps_and_cli#find-versions

borkdude16:12:58

I once made a library which may help a bit with this problem: https://github.com/borkdude/deps-infer

borkdude16:12:34

clj-kondo doesn't analyze deps.edn, project.clj or pom.xml, it just inspects .jar files and .clj files, so it has no idea which lib and version this is, but combined with the above you may be able to solve that problem

sheluchin17:12:21

Another piece to the puzzle! I will check out deps-infer in detail. What about inspecting the pom.xml in the jar files, is that something you would consider, or it's out of scope for clj-kondo? > Download an index of all of Clojars for inferencing of deps that are not in your local .m2/repository. This index must be kept up to date, e.g. daily, and be committed to some git repo where we can then fetch it. I need to look closer, but maybe https://github.com/phronmophobic/dewey can be used for this.

borkdude17:12:56

I would say it's out of scope / work that can be done in another tool, at least for now

borkdude17:12:27

unless it can be done in very pure way that doesn't add a lot of deps, etc. perhaps you can figure this out and if it's small enough, it could live in clj-kondo eventually

sheluchin12:12:19

Part of the problem I'm trying to solve is to be able to link namespaces to libs without downloading the libs. I'm not entirely sure this is even possible. Do you know? In the case of deps-infer, it works by scanning your .m2/repository, which obviously depends on libs being downloaded. The way I get around it now is by first downloading a given lib, running analysis on it to store all of its definitions, and then downloading one or more of repos that use the lib, and matching their usages' :to to the definitions of the lib. It's not a perfect process, but works well enough. I would just like it to work better 🙂

borkdude12:12:28

when you're analyzing a lib, isn't the lib already there?

sheluchin12:12:40

Yes lol it is. What I would like is to be able to look at any given repo and understand how its namespace usages map to the libs listed in one of its project files (deps.edn, project.clj, etc.). I'm looking to kind of invert the process I'm currently using, and not depend so much on analyzing repos in the context of a specific lib.

borkdude12:12:05

well, you can do this using deps-infer. start with a clean m2 directory (or set an alternative m2 directory). download deps. analyzes sources. run deps-infer and have the answer, kind of

sheluchin13:12:24

But that's back to downloading the libs, which is the thing I'm trying to avoid (which might be unavoidable). Unless I'm misunderstanding..

borkdude13:12:01

I don't understand, since you need the deps to analyze them in the first place

sheluchin13:12:34

Is it possible to infer what dep a required namespace points at without creating a runtime or running analysis on the full set of deps?

borkdude13:12:02

not without a global lookup table of all the deps in the entire universe that has once been populated

borkdude13:12:30

also note that deps can make up any namespace they want, so namespaces might not be globally unique

sheluchin13:12:58

I wish this were otherwise 🙂 like if required namespaces were transparent about their source dep... but okay, it sounds like I'd be better off starting off with creating that global namespace lookup table and then dealing with conflicts separately when trying to figure out where usages come from.

borkdude13:12:17

I once had a conversation in clojars, to populate such a global lookup table from all deps in clojars

borkdude13:12:22

by running deps-infer

borkdude13:12:40

you might able to find it in #C0H28NMAS and re-spawn that discussion

sheluchin13:12:59

Thanks for the feedback and pointers. I'm still not totally sure about the best strategy for me at the moment, but the mess in my mind will make sense after bit of time.

borkdude13:12:14

hammock time :)

sheluchin13:12:43

In case anyone comes across this thread looking for something related, I also found https://github.com/rads/deps-info, which is kinda related, but so far I haven't managed to squeeze much juice out of it.

borkdude13:12:09

a funny detail is that he called this library deps-infer too, but then I told him that I already had a library called like that ;)

sheluchin13:12:36

I don't know how you manage to do so much. There would probably be some interest if you were to share a little of your thought process and philosophy of development... although, you've probably already done that too 😛

borkdude14:12:55

just one step at a time :)