This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-01-31
Channels
- # announcements (22)
- # asami (19)
- # aws-lambda (4)
- # babashka (42)
- # beginners (43)
- # calva (28)
- # cider (1)
- # clerk (79)
- # clj-kondo (12)
- # clojure (47)
- # clojure-berlin (1)
- # clojure-brasil (1)
- # clojure-dev (12)
- # clojure-europe (40)
- # clojure-nl (2)
- # clojure-norway (5)
- # clojure-uk (3)
- # clojurescript (56)
- # clr (12)
- # conjure (8)
- # cursive (4)
- # datomic (78)
- # dev-tooling (6)
- # exercism (1)
- # fulcro (9)
- # hoplon (3)
- # jobs (3)
- # jobs-discuss (4)
- # lambdaisland (3)
- # leiningen (1)
- # london-clojurians (1)
- # lsp (125)
- # malli (32)
- # matcher-combinators (3)
- # nrepl (1)
- # off-topic (6)
- # pathom (39)
- # re-frame (13)
- # releases (2)
- # remote-jobs (3)
- # sci (7)
- # shadow-cljs (117)
- # sql (6)
- # squint (7)
- # tools-build (15)
- # tools-deps (12)
Has anyone has used clerk to generate their github readme? The idea being that I want to programmatically generate various sections in the readme (eg. screenshots, versions, links, etc.). I think I have an an idea of how to approach the problem, but wanted to check if anyone had already tried it.
That’d be awesome. I’ve made http://jsxgraph.mentat.orghttp://jsxgraph.mentat.orghttp://jsxgraph.mentat.orghttp://jsxgraph.mentat.orghttp://jsxgraph.mentat.org and other sidecar sites that kind of feel like READMEs but not the real thing
I didn't quite make it work, but I wanted to brain dump my thoughts while the context was still fresh.
Since my github readme was already in markdown format, I thought I'd try to take advantage of clerk's builtin support for markdown rather than a clj based notebook.
My intended approach was roughly
1) parse+evaluate the readme into a document
2) transform the document as needed
3) write ->markdown
which would be the equivalent of transform/->hiccup
Here's some scattered thoughts about the challenges I ran into:
• the clerk markdown parser doesn't keep around the tokens that are associated with the generated AST. It's not strictly necessary, but it would be useful when writing ->markdown
.
• the clerk markdown parser loses some information that would be nice for ->markdown
. For example:
sh
echo "hi"
The code type, sh
, is lost by the clerk markdown parser afaict.
• comments inside of code blocks show up as markdown rather than comments
• my readme is a way to demo the library and tends to have more side effects in top level statements. It would be idiomatic to put those side effects in a comment
block when inside a namespace, but it's less desirable inside a readme.
• A clj file corresponds a single namespace that executes from top to bottom. Code snippets in a readme don't necessarily run from top to bottom and can correspond to multiple namespaces/contexts.
• I would like to be able to add metadata to code blocks to control their execution environment, but I'm not sure what a good way to specify the meta data would be.
• It doesn't seem like https://github.github.com/gfm/#hard-line-break are supported
Not sure if this use case is one that clerk is interested in officially supporting. I'm still not sure I have a good idea exactly what an expected workflow might look like either. Anyway, just thought I'd share my experience in case it's helpful.Hi @smith.adriane, I have some draft of the function you mention n.markdown.transform/->md
in a PR against our markdown library: https://github.com/nextjournal/markdown/pull/6 see the notebook linked in the PR description for possible usages.
with that you could, do • parse and evaluate a clojure namespace • transform results in a md “readable” format (EDN blocks) • transform prose blocks back to markdown strings with a bit of gluing together these pieces you should achieve what you have in mind (If I understand correctly)
> the clerk markdown parser loses some information this is true in clerk, we should process fenced blocks with an explicit language as inert code blocks and not always as clojure code to be evaluated. Our markdown parser do have this information:
(nextjournal.markdown/parse "
sh id=xyz
code
") ;; =>
{:type :doc,
:content [{:type :code, :content [{:type :text, :text "code\n"}], :language "sh", :id "xyz", :info "sh id=xyz"}],
:nextjournal.markdown.parser/id->index {},
:toc {:type :toc},
:footnotes []}
> the clerk markdown parser doesn’t keep around the tokens that are associated with the generated AST if you mean the original markdown-it tokens (on which the parser is based) that’s true we don’t keep the original source strings from the tokens. We maybe could, but in the spirit of https://github.com/nextjournal/markdown/pull/6 you might thing of building a doc structure programmatically which doesn’t come from parsing markdown text, and you’d want to turn that structure back to a markdown string as well.
> It doesn’t seem like https://github.github.com/gfm/#hard-line-break are supported Thank you for reporting this! We’re currently ignoring hard break tokens
> comments inside of code blocks show up as markdown rather than comments
yes this is by design, in clerk to follow the convention that each form is a block per se. So each markdown fenced block is parsed as if clerk would parse a piece of clojure code, the resulting blocks are concatenated. We could maybe allow to opt out of this behaviour (?), or try to leave the parsing behaviour open.
I guess the functions from n.clerk.parser
could allow to build your own variant of document, which is (maybe) still compatible with clerk analyzer and eval.
> I would like to be able to add metadata to code blocks see the code snippet I pasted above, we parse fence info (pretty naïvely) according to https://github.com/nextjournal/markdown/blob/a84ab81bcd586b4360ba6b74a177aed9a9569499/src/nextjournal/markdown/parser.cljc#L77-L120 (this comes from previous work done for http://nextjournal.com) see the linked resources.
this should get hard breaks right https://github.com/nextjournal/markdown/commit/f3e40c37d312b0bd964cfe1660b3dd42a0f3068d, will propagate to clerk at a later point
Hi @U9EQP1K0X, thanks for the comprehensive response! Part of the thinking for what I am trying to do is that I have a bunch of .md files lying around where it would be nice to add small amounts of programmatic control. Since clerk has some support for md files, I thought I would try that, but it seems like the basic advice is to use clojure namespaces instead.
> Our markdown parser do have this information:
Ah! good catch. I was using nextjournal.clerk.parser/parse-file
and nextjournal.clerk.parser/parse-markdown-string
which seem to lose this context somewhere.
> (parser/parse-markdown-string
{:doc? false}
"
sh
foo
")
{:blocks
[{:type :code,
:text "foo",
:loc {:line 1, :end-line 1, :column 1, :end-column 4}}]}
There's some subtle differences between "code, with a little bit of prose" compared to "prose, with a little bit of code" that I'm not sure how to think about. I've found that Clerk does an excellent job for the "code, with a little bit of prose", but I'm still trying to figure out what kind of workflow I want to use for "prose, with a little bit of code". I know you can do that with clerk by using long comments, but there are some ergonomics issues that I'm concerned about.
> it seems like the basic advice is to use clojure namespaces instead or... if you have clerk process .md files the expected behaviour is the ~same as processing a clojure namespace > Code snippets in a readme don’t necessarily run from top to bottom and can correspond to multiple namespaces/contexts that would require that you take over the eval yourself (maybe), not sure what happens with code blocks declaring different namespaces
the problem i have with going full .md
is that clojure-mode in emacs won’t work anymore, so I’m now in a new environment
org-mode
was built for this! it’s still clunky but better, since you can pop in and out of Clojure mode
> the problem i have with going full .md
is that clojure-mode in emacs won’t work anymore, so I’m now in a new environment
I'm not sure what settings I've tweaked, but at least some features of clojure-mode
do work for me in emacs if the code is marked as clojure.
clojure
(+ 1 2)
For reference, I've been using markdown for my blog and run it through a similar process to clerk to generate the output, https://github.com/phronmophobic/blog/blob/master/markdown/dewey-analysis.md I was also wondering if I could throw all that code away and just use clerk.
oh, nice, I had in mind C-c C-k
to evaluate the full file, or namespace switching based on file etc
but maybe they just work??
I don't think it "just works". The behavior has always been a little clunky, but at least I get cider-eval-last-sexp
and syntax highlighting.
Hey all! I’m putting together an awesome-clerk
repo of links to cool uses of Clerk, notebooks etc… actually, just ANY uses, since I think any use at all is instructive. On the thread, if you’ve got anything public using Clerk (and wouldn’t mind me linking to it), would you mind dropping me a link?
https://www.juxt.pro/blog/using-clerk-for-aoc/ and the linked solutions? (maybe flag them as potential spoilers & incomplete)
I wrote a bit about rainbow tables a while ago. You can be the judge on whether it's up to your standards! https://github.clerk.garden/teodorlu/clerk-stuff/commit/7bd85d28726a0f166d8f4952b0dbf70936531b3e/src/rainbow_tables.html
haha I’m sure it is!! my standards are “does it use Clerk” so we can give folks lots of examples
I wrote some docs on styled text with clerk, https://phronmophobic.github.io/membrane/styled-text/index.html
@smith.adriane can I ask how you’ve set up Clerk build with deploy to Github pages? (I assume that’s what you’re doing, but I can’t find anything in https://github.com/phronmophobic/membrane/tree/master/.github/)
I don't yet, https://github.com/phronmophobic/membrane/blob/master/notebooks/paragraph.clj#L300 , but hopefully I'll have answer for you in a couple hours!
@U3X7174KS all of my recent libraries have a GitHub action that you can check out

https://github.com/mentat-collective/MathBox.cljs/blob/main/.github/workflows/gh-pages.yml
Basically build the site somewhere then point the final action at it. By default it deploys “public” and by default clerk builds to public/build so you probably need to add a directory argument at the end, Google the action and see what comes up
Perfect, thank you! Last time I used Github pages, I put static HTML files on the gh-pages
branch. Things have improved since then 😄
Haha that is basically what this thing does.. I think it might also ping GitHub to say that the new pages are ready?
I had started exploring the many, fantastic biochem-related linked data SPARQL endpoints and visualizing outputs with clerk. (https://github.com/zachcp/sparql-explore. https://zachcp.github.io/sparql-explore/) def a WIP though…..
Here is my documentation of clojure2d.color namespace with custom rendering. https://clojure2d.github.io/clojure2d/docs/notebooks/notebooks/color.html
Here’s a first pass at awesome-clerk: https://github.com/mentat-collective/awesome-clerk I searched Clojurians for any Garden links and added those, also added some of my docs notebooks plus other projects I’ve seen. I’ll organize these and write more, but keep them coming, and send me anything new as you publish it!
already found something useful I wasn’t aware of! thanks for putting this together ✨
once I get an “Advent of Clerk” style repo together for Structure and Interpretation of Classical Mechanics exercises I expect you all to read the full book 🙂

thx for this as it helped me find this: https://github.clerk.garden/behrica/vl-galery/commit/cbc1d1f044b2ac81a39453bf32f72cfce71d0b29/
been chatting with @borkdude about about reviving the https://github.com/nextjournal/clerk/issues/30 PR only to discover it already works (so long as you’ve previously had the clerk page open when online). 🌐
I noticed that clerk notebooks are slow to load and it's mostly due to a single request: https://storage.googleapis.com/nextjournal-cas-eu/assets/3J292uwuzpNFvuMeyWd1h7PtuAjBbUboke9HBHMhpRTTXYqgfxiaUGnx7396uTm879KgrLoMCFMF7qmdcHGAmw4f-viewer.js It takes about 4s just for that one file even though it's only about 2.5mb. Is that a non-eu thing or is that request slow for everyone (or just me)?
is that all network, or also loading the js?
just network
It's possible that it's just my crappy internet. I tried downloading it from an aws instance in california and it was much faster.
I'll see if I can check with someone else in california with residential internet
Yea, it's cached on second load. Just noticed that clerk pages often are slow to load.
I checked with someone else in california who has faster internet and the download was 1s. Much faster, but still slower than expected for 2.5mb. Hopefully, the CDN will make the difference 🤞 .
clerk’s js bundle now has cloudflare’s CDN in front of it if you’re using the latest main
. @smith.adriane can you try if it’s faster for you now?
@U5H74UNSF, my clerk notebook is now failing to build (I confirmed it still works with :mvn/version "0.12.707"
.
:git/sha "92cfcc59898c1aff24eef1b59ae07af8af0adac5"
is failing during analysis with the following error:
Class not found: (Class/forName "[B")
Which points to a required namespace, https://github.com/phronmophobic/membrane/blob/master/src/membrane/skia.clj#L550.@smith.adriane that looks familiar https://clojurians.slack.com/archives/CF595DNF7/p1676305329884289
Yea, that looks familiar
It was also an error with validate-tag
It seems like the recommended fix is to use extend
, right?
https://clojurians.slack.com/archives/CF595DNF7/p1676306544505019?thread_ts=1676305329.884289&cid=CF595DNF7 https://clojurians.slack.com/archives/CF595DNF7/p1676310240984099?thread_ts=1676305329.884289&cid=CF595DNF7
extend-protocol
expands into an extend call. I just macroexpand it to get the code that works.
ok, up and running again and it's much faster . thank you 🙏 !
OKAY, I haven’t updated the bigger docs on the project yet…
but I’ve just merged this and it’s cool so let me share.
These are drop-in replacements for build!
, serve!
and halt!
: https://github.com/mentat-collective/clerk-utils/blob/main/src/mentat/clerk_utils/build.clj#L48-L131
they work just like the old ones, except you can optionally supply a sequence of namespace symbols to :cljs-namespaces
, and you’ll get a This Just Works custom CLJS build, for both live development or static site builds

If you want to do this, I recommend adding a single cljs file, something like :cljs-namespaces ['my.code.sci-config]
, and then copying this namespace’s format:
https://github.com/mentat-collective/JSXGraph.cljs/blob/main/dev/jsxgraph/clerk_ui.cljs
any :classes
, :aliases
or :namespaces
you add here using this style will be available inside of your clerk viewers
proof that this works on GitHub Pages: https://clerk-utils.mentat.org
and on Garden… https://github.clerk.garden/mentat-collective/clerk-utils/commit/1681208f9af7e644897008e79dc9f5af64f53b69/
Phew, now time to go add “How to use this from Clerk” docs to all the libraries… and docs to THIS project stating what I’ve written above
cc @U098UL4QP, this will now be the easiest way to get your stuff going
I will try this out and get back to you. Thank you!
clerk’s js bundle now has cloudflare’s CDN in front of it if you’re using the latest main
. @smith.adriane can you try if it’s faster for you now?