Clojurians Log v2

hmm, https://cljdoc.org/builds/69371 I tried to push a new tag to specify clj only and the build is still failing on that one bit: https://github.com/mentat-collective/emmy-viewers/blob/cljdoc-v0.1.0/doc/cljdoc.edn

maybe the v is the problem there

but I feel like this has worked before?

@sritchie09, the cljdoc-<version> rebuild tag is currently for https://github.com/cljdoc/cljdoc/blob/master/doc/userguide/for-library-authors.adoc#overriding-articles-revision, not for API analysis.

got it

If I remember correctly, we didn't want to deal with the case of the jar sources possibly being out of synch with the git sources.

I'm trying to ramp up on how a cljs library can/should require an npm dep. It seems the topic is a bit complex? https://github.com/clojure/clojurescript-site/issues/224?

@sritchie09, still ramping up, how https://repo.clojars.org/org/mentat/emmy-viewers/0.1.0/emmy-viewers-0.1.0.jar, does ClojureScript learn about https://github.com/mentat-collective/emmy-viewers/blob/cfaa18473c9d45b74e3c02af83d7d981f799851c/package.json#L11-L52?

@lee each of the dependencies packages its own deps.cljs file; that’s how emmy learns about them, and emmy-viewers learns about them the same way

(vs emmy packaging a package.json)

Ok @sritchie09, to help my learning here, I'm going to do something simple. From a new empty directory, I'll create a deps.edn:

{:deps {org.clojure/clojurescript {:mvn/version "1.11.60"}
        org.mentat/emmy {:mvn/version "0.31.0"}}}

> clj -M -m cljs.main --repl-env node
ClojureScript 1.11.60
cljs.user=> (require '[emmy.value :as v])

Unexpected error (ExceptionInfo) compiling at (REPL:1).
No such namespace: complex.js, could not locate complex/js.cljs, complex/js.cljc, or JavaScript source providing "complex.js" in file file:/home/lee/.m2/repository/org/mentat/emmy/0.31.0/emmy-0.31.0.jar!/emmy/value.cljc

{:npm-deps
 {"complex.js" "^2.1.1"
  "fraction.js" "^4.2.0"
  "odex" "3.0.0-rc.4"}}

Good question, I have been in shadow land so long that I’m not sure if there are any preparatory commands to run to get that to work

I have avoided as much JS knowledge as possible, to my obvious detriment

I've not found clear docs on what works when/why/how. There is https://widdindustries.com/blog/cljs-npm-libraries, but maybe it is old when measured in JavaScript time?

I am working on static analysis for cljdoc, I can't remember if you generate any of emmy APIs at load time, if not it might be an easier way to go.

So I guess it needs :npm-deps true somewhere

Oh! Interesting. I'll toy with that to learn more. Apparently, https://clojurescript.org/reference/compiler-options#npm-deps.

That happened like 4.5 years ago, but it might be something the ragtag cljdoc team did not notice.

Haha it’s a clue anyway :)

Thanks for looking into this!! I can go clojure only too in the meantime

Hmm.... I'm not having luck with :npm-deps true nor :install-deps true.... but maybe I don't have the right incantation.

Maybe we should ask for help in #clojurescript

But... install-deps true does generate package.json, package-lock.json and node_modules.

❯ clj -M --main cljs.main -co '{:install-deps true}' -re node -r
ClojureScript 1.11.60
cljs.user=> (require '[emmy.value :as v])
WARNING: abs already refers to: #'clojure.core/abs in namespace: clojure.math.numeric-tower, being replaced by: #'clojure.math.numeric-tower/abs
Execution error (Error) at (<cljs repl>:1).
Cannot find module 'complex.js'
Require stack:
- /tmp/out361420216702976617719667604306161/emmy/value.js
- /home/lee/proj/oss/-verify/cljs-npm-deps/[stdin]

❯ ls
deps.edn node_modules  package.json  package-lock.json
❯ cat package.json 
{
  "dependencies": {
    "@cljs-oss/module-deps": "^1.1.1",
    "complex.js": "^2.1.1",
    "fraction.js": "^4.2.0",
    "odex": "^3.0.0-rc.4"
  }
}

Side thought: I wonder if some cljs libs these days require shadow-cljs.... (don't work with vanilla ClojureScript) and maybe should be analyzed with shadow-cljs... if that's possible...

I hope that’s not the case with this one… it is weird that Emmy works fine

Hmmm.... yeah, but my simple tests above were with emmy. This implies that emmy does not work with vanilla ClojureScript (or we just don't know how to get it to work with vanilla ClojureScript). But why does cljdoc cljs analysis pass for emmy? Dunno yet, that is cljdoc code I'm not familiar with yet.

Right, so https://github.com/cljdoc/cljdoc-analyzer/blob/2983ed108d9a44ab1c26cd0f4a0d85425893bb62/modules/metagetta/src/cljdoc_analyzer/metagetta/clojurescript.clj#L104-L128. This must be what makes emmy pass cljs API analysis. But for whatever reason, the trick does not work for emmy-viewers->emmy.

Ok, so yup, only immediate lib sources get the faked out js deps. So any js dep from emmy-viewers is faked out, but js deps from emmy are not.... and when emmy-viewers loads emmy, we get the no such namespace error on for example fraction.js/bigfraction.js.

So, @sritchie09 I shall raise an issue but an interesting discovery is that cljdoc cljs API analysis does not imply that a lib will work with vanilla ClojureScript. I think.

issue raised: https://github.com/cljdoc/cljdoc-analyzer/issues/84

Thank you for all the work here @lee , especially as I was distracted by family instead of helping - I’m glad that this failure makes sense and we’ve got a path forward

I’ve also got some work to do with testing vanilla cljs

I can't say I understand all the workings, but yeah, I think I have something to explore anyway! I'm gonna try, for my own education, emmy-viewers in a shadow-cljs repl.

Ok, cool, (as you already know), shadow-cljs handles all these deps.cljs automagically.

any ideas on how to resolve this one? Or why I’m not seeing it on the main Emmy build?

I can take a look sometime my-tomorrow @sritchie09

@sritchie09 a naive question from I guy who does not know: is :npm-depsin deps.cljs a https://shadow-cljs.github.io/docs/UsersGuide.html#publish-deps-cljs?

I think it’s a clojurescript thing too these days…

Vanilla that is

k, tx, will dig more tomorrow

@lee tasks complete: https://github.com/mentat-collective/emmy-viewers/pull/56 I want to get a couple more PRs in today, then I’ll cut a release

Coolio. I just double checked that I did not steer you wrong by: 1. cloning emmy-viewers, 2. then rm -rf ~/.m2/repository/org/mentat 3. then from emmy-viewers dir, clojure -T:build install, 4. then from cljdoc-analyzer dir:

❯ clojure -M -m cljdoc-analyzer.cljdoc-main '{:project "org.mentat/emmy-viewers", :version "0.1.0", 
  :jarpath "/home/lee/.m2/repository/org/mentat/emmy-viewers/0.1.0/emmy-viewers-0.1.0.jar",
  :pompath "/home/lee/.m2/repository/org/mentat/emmy-viewers/0.1.0/emmy-viewers-0.1.0.pom",
  :languages nil, :repos {"clojars" {:url ""}, "central" {:url ""}}}'

@sritchie09 unless you truly only want your clj API analyzed, you'll probably want to turf this: https://github.com/mentat-collective/emmy-viewers/blob/main/doc/cljdoc.edn

Oh yeah I forgot, that was a workaround I had tried

@sritchie09 I've hacked past (but have not yet pushed changes) to handle the JavaScript requires issue. But, of course, there is always the next issue! Cljs analysis is now failing with:

No such namespace: nextjournal.clojure-mode, could not locate nextjournal/clojure_mode.cljs, nextjournal/clojure_mode.cljc, or JavaScript source providing "nextjournal.clojure-mode" (Please check that namespaces with dashes use underscores in the ClojureScript file name) in file file:/home/lee/.m2/repository/io/github/nextjournal/clerk/0.14.919/clerk-0.14.919.jar!/nextjournal/clerk/render/code.cljs

My next error is:

No such namespace: lambdaisland.deep-diff2.diff-impl, could not locate lambdaisland/deep_diff2/diff_impl.cljs, lambdaisland/deep_diff2/diff_impl.cljc, or JavaScript source providing "lambdaisland.deep-diff2.diff-impl" (Please check that namespaces with dashes use underscores in the ClojureScript file name) in file file:/home/lee/.m2/repository/djblue/portal/0.42.1/portal-0.42.1.jar!/portal/ui/viewer/diff.cljs

Oh interesting… okay yeah I think I have a bunch of “provided” dependencies that only come into play if you pull in a special dependency of clerk

I need to do some class path hygiene and think about what I’m actually shipping I think

Hmm... I do see deep-diff2 as a provided dep of https://repo.clojars.org/djblue/portal/0.42.1/portal-0.42.1.pom. But, I think provided deps only apply to a library, not its dependencies.

I'll muck around a bit more and contrive emmy-viewers pom and jar. I'd like to see emmy-viewers analyzing, at least locally, on my dev box.

Also: not sure if you want that /demo dir in your jar. Maybe your users need it?

I have been migrating code out of those final two namespaces, but you’re right that they don’t need to be in the jar, just on the class path for a couple of the demos that can’t use the proper API yet

I will move those out

Oh wait, https://github.com/mentat-collective/emmy-viewers/blob/27675959bafbef23c7e5dc91f1ded75bcd7d36a7/src/emmy/viewer/sci.cljs#L2-L3

Yeah but I can remove them from there and only add them in for the GitHub pages build

Ok, I have emmy-viewers docs building locally. Summary of hacks: 1. have cljdoc-analyzer fake out JavaScript requires found on classpath (I'll clean this up and push probably sometime today) 2. emmy-viewers jar a. delete /demo dir b. edit /emmy/viewer/sci.cljs to comment out references to demo namespace 3. emmy-viewers pom, added provided deps:

<dependency>
  <groupId>lambdaisland</groupId>
  <artifactId>deep-diff2</artifactId>
  <version>2.8.190</version>
  <scope>provided</scope>
</dependency>
<dependency>
  <groupId>hickory</groupId>
  <artifactId>hickory</artifactId>
  <version>0.7.1</version>
  <scope>provided</scope>
</dependency>

Amazing! I can take care of 2 and 3. Thank you @lee !

For you @sritchie09? Always a pleasure!

I'll letcha know when I've pushed item 1. When the dust settles, I'll likely add emmy-viewers to our cljdoc analyzer integration test suite.

Ok, @sritchie09, fix for item 1 is live. Lemme know when you address items 2 and 3 (and a release), and then I'll add emmy-viewers to the cljdoc-analyzer integration test suite.

Thank you, will do!