I actually thought a lot about this during parinfer development

comments in the file were getting pretty long, and I felt that I could go the literate-programming route

I ultimately decided to keep the file with minimal comments

and instead opted to create a code.md that has a design-level narrative separate from the code, but actually links directly to functions and vars by name

https://github.com/shaunlebron/parinfer/blob/master/lib/doc/code.md

we're doing something similar for the doc build system on my project at work

@mattly: is it open?

no

I think code.md has the documentation aspect of literate programming. But the tangle/weave aspect is useful, the order of understanding a program is not necessarily the order the program must be written in.

I suppose if you start at code.md, it has the same effect :thinking_face:

in our case the doc system is more meant to tell a story about data as it flows from "a user did x" to "here's how it appears in the reports"

so it's like "here's how we ingest this patient data event", then "here's how it passes through ETL" and finally "here's how the reports and analytics tools use that in queries"

and the doc system pulls in from multiple code repositories to do that

@mattly: that sounds really good

i struggled at Stripe to understand the data flow between three iframes in our Stripe.js v3, and thought about writing something to document it in a similar way

@shaunlebron it's worked well for us, and helps us keep even non-engineer eyes on the process

nice

because if there's a bug in that flow, no one is ever going to know from looking at the output

more tools like that should be made

yes!

I'm pushing to extract parts of the system and OSS it, but in all reality I think we're a year off from having the headspace to do that

sounds like a big undertaking, honestly may be too specific for others?

a blog post would be great though

yeah, that's part of it too

just to show principles in action

hm, yeah, I'll bring that up

@dominicm without reading the knuth’s paper, I’ve thought for a while that docs are a bit like tests, in that they fulfill a separate narrative and that they have to be written in parallel to—and in sync with—code

maybe that’s not at all what he’s talking about, but I’m interested to see if they can really be unified in “literate”. But apart from execution order vs. reading order that literate addresses, I would go further and say that there is also a problem of certain reader abstractions not mapping 1-to-1 with the abstractions created in the code

(this is all very hand-wavy in my head though)

@shaunlebron I agree with all you are saying. I think it takes a lot of discipline in order to sync docs with code though. Knuth admits that he poorly documents code usually, but that his tool forces him too. He also created WEB, so it's no surprise he feels ecstatic when using it.

i wanted to see an example of WEB in action. found tex.web rendered as a pdf here: http://brokestream.com/tex-web.html

in risk of getting ever-more off-topic for the channel, but chasing the topic-du-jour, do you know of any other kind of lightweight "text to various render methods" akin to say, asciidoc or org-mode but that give you more free-form elements for creating document strucutre?

@mattly what do you mean by: > free-form elements for creating document structure asciidoctor supports a generic block, that can nest with different numbers of ----, I think that's what you mean, but uncertain.

hm. I guess what I mean is that I'd like to be able to register with the parser/renderer "block types" that get special treatment in the rendering process

You can do that

There's extensions & stuff. Very extensible

I'll take a deeper look at it, thanks

clojurescript unraveled uses it: https://github.com/funcool/clojurescript-unraveled

oh, asciidoc has stem blocks that are more like what I want