Literate execution/first-class comments

[rolyp]

• Mock up (as a .fld file) a program containing such a comment, illustrating:
  • Escaped Fluid code that evaluates to a string
  • Escaped Fluid code that evaluates to a visualisation
  • Scope of comment relative to e.g. function arguments
• Think about how this might interact with desugaring if comments are on surface syntax

---

High-level issue for developing further the idea of “transparent text” so that narrative text can be arbitrarily interleaved into the (execution of the) program. Currently we support LinkedText as an output, but how can we use something similar to attach narrative to intermediate values? We want to be able to “unfold” or expand a subcomputation and potentially see text there as well, but it’s not clear how that works unless linked text can be attached to arbitrary intermediate expressions. Perhaps LinkedText should evolve into a kind of structured comment that can contain embedded expressions that are used to compute some of the comment (currently just text, but potentially graphics as well). Something like literate programming but where some of the surrounding documentation is being generated at run-time.

For example, consider an expression e which computes an intermediate data structure as part of a data analysis pipeline. One could attach a structured comment to e which documents that step of the pipeline using a mixture of:

1. natural language;
2. escaped Fluid expressions, which are resolved at runtime in the execution context of e, and evaluated to either:
   • LinkedText (natural language computed from data)
   • more generally, an arbitrary View which can be rendered inline as part of the comment

Supporting arbitrary embedded views might point to how we can attach custom visualisations to intermediate values.

See also:

• Captions as LinkedText #1182
• Paragraph: interpolation syntax #1187
• Explore intermediate values #270

[rolyp]

@JosephBond I think this may be the first thing we need to have a bit of a chat about. Might be good to involve @dorchard in this conversation at some point as I think he is interested in something similar.

[JosephBond]

I am taking the opportunity to read the original paper by Donald Knuth, found here, and I've also found this paper, which seems relevant to what we want to do in this space

[JosephBond]

Related Work

Emacs Org-Mode

Has a very similar aim to our idea, it's meant to promote literate programming and reproducible research, by allowing a user to embed code from multiple languages, as well as meta-data into a text file. I believe that the difference is in the context: org-mode allows a user to embed code from multiple languages into their document, but it requires the use of emacs, and has (afaik) no interactive features (incidentally, I wonder how difficult it would be to embed fluid's interactive features into org-mode?) In our case, I think the writing portion needn't necessarily be tied to any single tool, although we are targeting VS code, and we will only be supporting the use of fluid as the language. That should mean we can provide smoother support for fluids extra features like interactive visualization. Further, org-mode only allows for the inclusion of plain text. I wonder if we can differentiate ourselves from it by allowing the user to toggle the view of a visualization/value/piece of code (unsure what the distinction will be here anymore), within the view. Perhaps we could tightly incorporate the fluid runtime into the editor interface?

Literate Haskell / Haddock

Haskell has 2 tools that are somewhat related to what we are trying to achieve, literate Haskell, and Haddock. My limited understanding is that Literate Haskell has fallen out of favour, but Haddock has been widely adopted.
Literate Haskell is equipped with two styles, "bird style" and "block style". This stackoverflow comment by Norman Ramsey provides some insight as to an experienced academics usage of these tools. Perhaps worth contacting him as a collaborator?