Fork me on GitHub
#cljdoc
<
2021-11-22
>
Matthew Davidson (kingmob)21:11:53

Were there any further updates on the error complaining about a missing cli-matic? I just tried running locally with Docker, and got the same error Toby did a month ago:

~/Documents/Code/manifold |master:zap:*
❯ docker run --rm \
        --volume "$HOME/.m2:/root/.m2" \
        --volume /tmp/cljdoc:/app/data \
        --entrypoint clojure \
        cljdoc/cljdoc -M:cli ingest \
          --project manifold/manifold \
          --version 0.2.0
Syntax error (FileNotFoundException) compiling at (cljdoc/cli.clj:1:1).
Could not locate cli_matic/core__init.class, cli_matic/core.clj or cli_matic/core.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

martinklepsch21:11:55

could it be that you have to use -A: due to an older clojure CLI version?

martinklepsch21:11:28

docker run --rm \
        --volume "$HOME/.m2:/root/.m2" \
        --volume /tmp/cljdoc:/app/data \
        --entrypoint clojure \
        cljdoc/cljdoc -A:cli ingest \
          --project manifold/manifold \
          --version 0.2.0
like this

Matthew Davidson (kingmob)21:11:45

Well, I can check real quick…

Matthew Davidson (kingmob)21:11:00

That worked to trigger a download of cli-matic, though it also warned me -A was deprecated.

Matthew Davidson (kingmob)21:11:32

But I downloaded the Docker image for the first time just a couple hours ago

martinklepsch21:11:38

And did it work in running the ingest?

Matthew Davidson (kingmob)21:11:04

(It’s not building my latest local version properly, but that’s at least a new error. It’s building something.)

Matthew Davidson (kingmob)22:11:31

Oh fuck. No wonder my build’s not picking up the new namespace. manifold has a special override in https://github.com/cljdoc/cljdoc-analyzer/blob/master/resources/config.edn

martinklepsch22:11:04

oh yeah, that would do it

martinklepsch22:11:15

that’s kind of an awful thing to debug, sorry you ran into that!

martinklepsch22:11:01

if you use a different group/artifact id that should solve the issue

martinklepsch22:11:17

and of course we can extend what’s there / remove the special config if that makes sense

Matthew Davidson (kingmob)22:11:49

Ahh, it wasn’t so bad, I already had my suspicions due to a few oddities here and there. We talked about updating the group ID, but decided against changing it out of historical inertia. If it’s ok with you, I’m prepping a PR for cljdoc to remove the custom handling for manifold, and I’ll modernize it to support cljdoc directly

martinklepsch22:11:23

That sounds great!

Matthew Davidson (kingmob)22:11:05

Thanks. Hmm, looks like there’s special handling in both cljdoc and cljdoc-analyzer

martinklepsch22:11:23

Feel free to open an issue if its too much

Matthew Davidson (kingmob)22:11:14

Thanks, that’s pretty nice of you. I think it’ll just take a bit of back-and-forth, since cljdoc refers to a SHA of cljdoc-analyzer.

Matthew Davidson (kingmob)18:12:38

Thanks for your help, @U050TNB9F, the new docs look great

🚀 1
martinklepsch19:12:21

@kingmob nice! do you want to share a link here? always love to see some things to celebrate in #cljdoc

Matthew Davidson (kingmob)20:12:13

https://cljdoc.org/d/manifold/manifold/0.2.3/doc/read-me It’s not that different from before, I guess, other than the new ns, but cljdoc makes everything look good 😊

Matthew Davidson (kingmob)10:08:49

OK, slight update to the missing cli-matic issue. It just happened again, and NOT on manifold. I was updating the docs of my True Grit library, and ran into it again. Manifold shouldn’t be anywhere on the path. Switching from -M:cli to -A:cli once again triggered the cli-matic download and fixed it so I could use -M:cli , but it’s not a Manifold-specific issue, apparently.

lread11:08:25

Hiya @kingmob, I'll catch up on this thread in a bit and try to help.

lread11:08:35

@kingmob, is it reproducible? Can you give me the exact command you ran?

lread12:08:19

I run cljdoc locally a bunch from docker and haven't yet seen this error. What OS are you on?

lread12:08:03

btw fyi: noticing that manifold jars scm is pointing back to https://github.com/KingMob/manifold. Is this intentional? Don't you really want: https://github.com/clj-commons/manifold?

Matthew Davidson (kingmob)12:08:51

@UE21H2HHD I'll take a closer look when I'm back at my laptop later. For now, I think the issue is an initialization one. If you destroy you clj doc docker image, start from scratch, and use one of the docker commands with the -M:cli flag, I suspect that will trigger it

Matthew Davidson (kingmob)12:08:59

Once cli-matic downloads once via -A, then -M never has any further problems.

Matthew Davidson (kingmob)12:08:33

Yes, the manifold SCM is wrong, but I was waiting until the next release to fix it

lread12:08:38

Interesting and weird. Will try try to reproduce.

Matthew Davidson (kingmob)12:08:19

I'll send the precise command later.

lread12:08:44

Cool, thanks.

lread12:08:36

> Yes, the manifold SCM is wrong, but I was waiting until the next release to fix it A while back, slipset worked out a flow for deploying from ci, if you are interested I'll find it.

lread13:08:01

Back to your issue… I wiped out all of my docker images via:

docker system prune -a --volumes
(drastic, eh? simple_smile) Then I tried an ingest:
docker run --rm \
  --volume "$HOME/.m2:/root/.m2" \
  --volume /tmp/cljdoc:/app/data \
  --entrypoint clojure \
  cljdoc/cljdoc -M:cli ingest \
    --project org.cljdoc/cljdoc-exerciser \
    --version 1.0.77
And it seemed to work fine. I’ll poke around a bit more. Two ideas: 1. there was a concurrent download bug in clojure tools deps… not sure if that was fully fixed. 2. maybe some confusion between docker .cpcache and mapped .m2 volume.

lread13:08:07

Seems like it is idea 2!

$ rm -rf ~/.m2/repository/cli-matic
$ docker run --rm \
  --volume "$HOME/.m2:/root/.m2" \
  --volume /tmp/cljdoc:/app/data \
  --entrypoint clojure \
  cljdoc/cljdoc -M:cli ingest \
    --project org.cljdoc/cljdoc-exerciser \
    --version 1.0.77
Execution error (FileNotFoundException) at cljdoc.cli/eval138$loading (cli.clj:1).
Could not locate cli_matic/core__init.class, cli_matic/core.clj or cli_matic/core.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

Full report at:
/tmp/clojure-4303972953105411104.edn

lread13:08:08

Adding an -Sforce to recompute the classpath seems to do the trick.

$ docker run --rm \
  --volume "$HOME/.m2:/root/.m2" \
  --volume /tmp/cljdoc:/app/data \
  --entrypoint clojure \
  cljdoc/cljdoc -Sforce -M:cli ingest \
    --project org.cljdoc/cljdoc-exerciser \
    --version 1.0.77
Downloading: cli-matic/cli-matic/0.5.4/cli-matic-0.5.4.pom from clojars
Downloading: cli-matic/cli-matic/0.5.4/cli-matic-0.5.4.jar from clojars

lread14:08:03

I’ll create a git issue for this, likely just a doc update, but want to think about options.

lread14:08:22

Ok @kingmob, issue created https://github.com/cljdoc/cljdoc/issues/654, I’ll likely proceed with my proposal of adding -Sforce to docker cmds in the running locally docs.

Matthew Davidson (kingmob)14:08:00

Here was the command that gave me errors:

❯ docker run --rm --volume /Users/matthew/Code/TrueGrit:/repo-to-import --volume /Users/matthew/.m2:/root/.m2 --volume /tmp/cljdoc:/app/data --entrypoint clojure cljdoc/cljdoc -M:cli ingest --project net.modulolotus/truegrit --version 1.0.10 --git  --rev b191a267cac2566ae3198106086df5d9d87f0d60

Execution error (FileNotFoundException) at cljdoc.cli/eval138$loading (cli.clj:1).
Could not locate cli_matic/core__init.class, cli_matic/core.clj or cli_matic/core.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

Matthew Davidson (kingmob)14:08:42

Sounds like you have an idea of what was going on, though

lread14:08:38

Thanks for the repro! Ya think I get what is going on.

Matthew Davidson (kingmob)14:08:14

Couple other things: I’m familiar with Erik’s bb deploy script, and we use it for some of Zach’s other old libs, but Manifold and Aleph are still on their own maven group for historical continuity reasons, so it won’t work with them at the moment (lacks the credentials).

Matthew Davidson (kingmob)14:08:06

I’m not yet aware of any great way to change the group ID in a seamless manner

Matthew Davidson (kingmob)14:08:34

Glad to hear the bug is found, btw. Lmk if you need anything on my end

lread14:08:15

Oh! Interesting. Yeah there is currently no way to seamlessly move to a new group-id, but there are https://github.com/clojars/clojars-web/issues/801.

Matthew Davidson (kingmob)14:08:58

Oooh, one suggestion, while it’s on my mind: As a library author, it’s hard to weave in and out of the https://github.com/cljdoc/cljdoc/blob/master/doc/running-cljdoc-locally.adoc file, since half of it applies to me, and half doesn’t. I think it would be better split into 2 docs, one for lib authors, one for cljdoc devs. Even with a little redundancy between docs, I think it would be way easier to follow.

lread14:08:01

For rewrite-clj the original author granted me admin access to clojars rewrite-clj so that I could generate deploy tokens.

lread14:08:38

Yeah @kingmob I agree, thanks for bringing that up. I really struggled when I wrote that doc. And I struggle when I read it. simple_smile I frankly find the whole previewing locally workflow overly complicated, but we can start with making existing docs easier to follow. I’ll create a separate git issue.

Matthew Davidson (kingmob)14:08:19

Yeah, Zach gave me all the perms I need on clojars, it’s just that it’s a bunch of work, and potential chaos, all so people can type org.clj-commons/manifold instead of manifold/manifold 😅 It always feels like there’s more important stuff to tackle. I could probably get deployment working with Circle without changing the group id, though

Matthew Davidson (kingmob)14:08:56

If it helps any, I have some build-clj commands to help get cljdoc running, maybe a gist or some official snippets in the docs could help people out. Plus there’s the whole “Do I push to Github or not?” question

lread15:08:47

But you are still deploying to manifold/manifold on https://clojars.org/manifold? No need to switch to org.clj-commons/manifold ?

Matthew Davidson (kingmob)15:08:26

Yep. Changing the maven group id is pretty low on my todo list.

lread15:08:56

Yeah, and disruptive for users of manifold, so I wouldn’t even recommend it!

Matthew Davidson (kingmob)15:08:07

If there were a great way to let casual dependents know about upcoming changes to aleph/manifold’s group id, it’d be more compelling, but… ¯\(ツ)

lread15:08:33

Ya, true!

Matthew Davidson (kingmob)15:08:35

@UE21H2HHD I’m gonna sign off for the night (2300 here). Thanks for all your help!

lread15:08:21

My pleasure, and if you need any help in getting your docs looking great on cljdoc, don’t be shy to ping us!

Matthew Davidson (kingmob)07:08:40

Looks good! Next step is to remove any reason to publish, git push, or mvn install. 😉 Being able to run cljdoc straight on the files would be ideal.

lread11:08:22

Working directly on files would be convenient, but different from how cljdoc production works today. This might become more natural/possible if/when I get around to supporting git-hosted repositories. Not sure how we can avoid the git push if you want to preview that your links back to GitHub work. At one point I was toying with a 2nd docker image to simulate a GitHub repo locally. That just all got too complicated.

👍 1