First-class pandoc/citation support?

[jzelner]

I'm loving using using this for making interactive tutorials for teaching, but the lack of support for pandoc-style citatons+bibtex was a bit of a pain point at first. Simply running pandoc on the file doesn't work for two big reasons:

1. Pandoc strips out the yaml headers when formatting the file and,
2. Pandoc turns ```js fenced code blocks into ``` js (i.e. w a space added between the fence and the language ID), which causes observable to treat it as a display rather than executable.

I have dealt with this myself by writing a short/hacky R script (below) that uses pandoc to process the MD file w/the citations in it and spits out a commonmark-formatted file that has the citations formatted as HTML in the text and then copies over the yaml header from the original file and prepends it to the output:

#! /usr/local/bin/Rscript

read_metadata <- function(file) {
    x <- readr::read_lines(file)
    rng <- grep("^---$", x)
    x <- c(x[rng[1]:rng[2]], rep("", 1))
    return(x)
}

a = commandArgs(trailingOnly = TRUE)
print(a)
outpath <- "docs"
fname <- basename(a)
outfile <- file.path(outpath, fname)

cmd <- sprintf("pandoc %s -t commonmark --citeproc --csl=aje.csl --bibliography=contextual-epi.bib -M link-citations=true| sed -e 's/\ js/js/g'",a)

header <- read_metadata(a)

x <- system(cmd, intern = TRUE)
x <- c(header, x)

write(x, file = outfile)

Right now I have this automated using chokidar (in a script called citewatch.sh to watch the raw source directory):

#!/bin/bash
chokidar rawdocs/*.md -c "scripts/makerefs.R {path}"

And I modified the npm run dev command in package.json to start both this script and the preview server at the same time using concurrently:

    "dev": "concurrently --kill-others \"observable preview\" \"./citewatch.R\""

This all works pretty well, but it would be nice to have it work more seamlessly in framework if possible, particularly so that the processing of the citations could happen somewhere along the buildchain from docs directory rather than doing what I have done, which is writing the processed file into docs to trigger the dev server to run/reload.

I know there is a pandoc plugin for node (https://github.com/eshinn/node-pandoc) but it is a big aged at this point, and the pandoc citation solutions for markdown-it are also somewhat limited. I looked at citation-js but it involved plunking a lot of js into the markdown and also live-rendering citations, which I didn't need.

In any event, having this be part of the package for framework would, I think, really increase its uptake w/academic types and make it a nice alternative to quarto for making slick, data-driven interactives.

[CAYdenberg]

I had a random idea to use citeproc-js from a component to render citations on the page, together with a custom theme/stylesheet that would print the page (directly from the browser) in a format acceptable for submission to a journal. Would a component avoid "plunking a lot of js into the markdown" - ie be a way to reuse javascript logic to correctly format?

[jzelner]

Hm - I guess I don't know enough about components to say one way or the other. To me, it's mostly a question of being able to include a citation that looks like "So and so said [@foo2022]" in the text and have it render into something that has the citations properly formatted and has a bibliography that is built off of bibtex (or equivalent) and a csl template. I

[CAYdenberg]

I think you could do that with a component? ... you would also (I guess) have to dump all the citations into a variable or generator so you can output the bibliography later.

[jaanli]

+1 on this!! Would really benefit from this for all the work I’m doing that requires observable framework.

[jzelner]

Just seeing if we could bring some version of this back to life?