Clojurians Log v2

With the recent version of clay (https://clojars.org/org.scicloj/clay/versions/2-beta56), it's now possible to publish notebook style documentation to cljdoc. The :gfm format target produces markdown and SVG visualization provided in hiccup format will be rendered as static SVG files and included as images. For an example, see https://cljdoc.org/d/org.soulspace/qclojure/0.17.0-SNAPSHOT/doc/tutorial. Thanks @timothypratley for making this happen. gratitude

The ability to incorporate HTML documentation has come up before, as I recall (I think for Marginalia, at least). Let me see if I can find the issue for that...

For security reasons, my thought was that user-provided JavaScript running on cljdoc was not a great idea. But maybe there is some safe way to do it.

(I didn't find a relevant issue -- there was a Marginalia-specific issue but that was more about literate programming than adding HTML pages)

Clay can generate Markdown tho' can't it? Maybe that's a path to get Clay-generated tutorials into cljdoc?

We certainly discussed it here, @seancorfield... https://clojurians.slack.com/archives/C8V0BQ0M6/p1699334728383859

Ah, but the HTML doc part never made it into an issue, just the Marginalia-specific aspect... I remember the discussion (vaguely) but hadn't gone looking for it here.

Clay can produce *.qmd files (for input to Quarto, I think?). Which aren't "real" Markdown files that cljdoc could understand (and there's JS files generated too, so that gets us back to the security issue above and really the general issue of embedding HTML etc into cljdoc stuff). 😞

some supported kinds for showing output rely on JS, for example vega output, I suppose. Currently I have nothing in the tutorial where JS is necessary, the output is SVG wrapped in kind/html. I understand your security concerns, but maybe there's a way to get a subset of clay supported. At least we have all parts of the picture under control. Can we support different kinds as "embedded formats" (e.g. SVG, or TeX) in markdown? Maybe as fallback by rendering them as images? Going that route would at least address the HTML problem.

If the JavaScript is not generated, we could have cljdoc include that static JS (as needed) for clay articles. I think cljdoc supports SVG... It might support Tex because https://cljdoc.org/d/lread/cljdoc-exerciser/1.0.57/doc/document-tests/asciidoctor-features#_math, if that's relevant. QMD is an extension of MD, so we could, technically, support it through custom extensions.

Not sure if it is relevant, but cljdoc uses an older version of MathJax (v2.7.9), because that is what AsciiDoc requires.

maybe we can check with Daniel, he should know. I've notified him about this thread.

but I really think, having clay tutorials on cljdoc is a nice additional way to document libs.

To gauge general interest, I'd love for others to chime in if they are interested in this feature.

Having been recently exposed to Clay -- and publishing the transducers article on Civitas -- I can definitely see value in being able to write docs that way and have them included in cljdoc. Could cljdoc include a Quarto step to handle those .qmd files perhaps? So the workflow for users would be to run clojure -M:clay -A:markdown and commit the generated .qmd files (under docs/, alongside any regular .md files), and then cljdoc would use Quarto on those as part of generating the HTML etc?

Without understanding specifics of Clay, from a cljdoc perspective, that seems like a good approach to me.

Thanks for this discussion, very encouraging. A few of the relevant visualization kinds can be rendered in a way that will not require Javascript, and in principle, Clay should eventually support such Javascript-free targets. We are not sure when this will happen, though. (In general, the future of Clay is to replace its rendering engine with https://github.com/scicloj/kindly-render by @timothypratley. There, we have a modular system to support various target formats. Supporting different markdown flavors, as well as Javascript-free HTML, has been a goal. As far as I know, Timothy did have cljdoc as a future use case in mind. At the moment, we are unsure about our capacity to develop these additional features in the near future. None of it is a lot of code, actually, but there are some delicate details of compatibility that will need some careful thinking.)

I'll be glad to keep thinking about it. 🙏

In the meantime, we could experiment with an ad-hoc solution on the Clay side that will create the desired flavor of markdown and just work with the visualization kinds which are currently needed. But maybe we should think a bit about what it requires, to see how much of a tech debt it might be.

I'm not in favor of solutions that add to the technical dept. I wouldn't rush things but wait for a well designed solution.

Coolio, when you are ready, we can explore how cljdoc might someday support Clay.