Make it easy to use downlit from html_document()

[hadley]

So you can easily get R powered syntax highlighting and autolinking.

(And probably also worth supporting in github_document() and html_document())

[maelle]

Why not in bookdown too? 🙂

[cderv]

(And probably also worth supporting in github_document() and html_document())

@maelle If supported in rmarkdown output format, it should be supported in bookdown too. (gitbook() inherits from html_document())

[apreshill]

Currently implemented in the development version of distill: https://github.com/rstudio/distill/blob/master/NEWS.md#distill-v09-development

rstudio/distill#165

[cderv]

I started to look into this to get a sense of how it works and what we could do.

We could offer a highlight_downlit option as done in distill so that the user can activate downlit highlighting and autolinking if desired (it would be FALSE by default). This would allow to have downlit only as Suggest in rmarkdown. if highlight_downlit=TRUE, we would check if installed and set to FALSE if not.

Re. how : There is downlit::downlit_html_path() or downlit::downlit_md_path() that would fit ok as postprocessors, or just as pre processor for md file. However, I had a look at hugodown and distill. Both are using a knitr hooks that does the source highligth, and hugodown does also the output highlight and the linking of inlines with its internal link_inline function. (while distill does not it seems - or I missed it).

I tend to find the use of the downlit_*_path less intrusive for rmarkdown and they work like a filter, usually done with Lua. But, both functions do not seem to work the same currently: html one use pandoc classes by default and insert the class downlit to pre tag, while md one use chrome classes and insert chroma classes. chroma is rather for hugo, right ?
Currently we can't change the classes use in those function.

Have I missed something ? Should we adjust those two functions to be more the same ? or should I go the distill way and build with knitr to apply downlit transformation ?

[hadley]

You can supply the classes argument to downlit_*_path() to change which classes it uses for syntax highlighting. But I'm also happy to make any changes needed to make downlit easier to use in rmarkdown itself.

[cderv]

downlit_md_path() does not have one yet but yes downlit_html_path does.

I need to understand better where to when to apply downlit processing to be sure we keep other Pandoc highlighting.

• Applying on input md seems the easiest but it would modify the AST for later Pandoc processing as CodeBlock are transformed to raw html. we would use downlit_path_md but it requires to provide classes_pandoc instead of classes_chroma maybe.
• Applying after would require to deal with how Pandoc does the highlighting. If we deactivate the highlighting, other language loose it.
• We could apply at knitr stage with hooks, but we would still have raw html in the Pandoc AST I think.

I'll keep testing. As rmarkdown format are for more general use than specific one like hugodown::md_document, we need to be sure activating downlit won't have side effet on how Pandoc works. Adding to github_document though seems safe.

[atusy]

It sounds tough to take care of Pandoc AST...
If I were to, I would try post-processing, and limit highlighting to default which uses highlight.js.
In this way, we can support highlighting on other languages.
We can think about other highlighting themes later.

[maelle]

Or to prevent Pandoc from highlighting R code blocks, rename R to notAnActualProgrammingLanguage. 🙊 Then downlit could highlight them.

[cderv]

IMO there is not really an easy way to support markdown output. Syntax highlighting with HTML in markdown output would depend on the HTML renderer when previewed. It works for hugodown::md_document because it aims at hugo support with chroma.
For Github, the same could have been possible but from my test, we cannot insert HTML with class of nodes - they are all stripped out by Github previewer. so, we cant insert generated HTML by downlit in a md document and keep the highlighting by Github. If we post-process with downlit_md_path(), the resulting markdown file will contain some unuseful html code. Github will preview the file to HTML correctly an the link introduced by downlit will be kept but we loose syntax highlighting.

I believe we should just support HTML document for now. That is what the PR is doing.

[hadley]

Agreed; adding downlit to md_document() isn't correct because it can be used for many formats other than html, and hugodown's use is a special case.

[cderv]

TIL: Pandoc highlighter does not understand correctly the R native pipe. This leads to not so nice highlighting in html_document()

---
title: test
output: 
  html_document: 
    highlight: pygments
---

```{r}
f <- function(x) { x + 1 }
2 |> f()
```

```{r}
(\(x) x + 1)(2)
```

I believe this would require to teach Pandoc syntax highlighter and / or theme.

Using downlit helps with this, and would be working with any Pandoc version. Example from PR #1941
Same doc gives better result (not red operator is better).

---
title: test
output: 
  html_document: 
    highlight_downlit: TRUE
    highlight: pygments
---

```{r}
f <- function(x) { x + 1 }
2 |> f()
```

```{r}
(\(x) x + 1)(2)
```

Noting this here to not forget I looked into this.

[cderv]

It should now be possible to use downlit syntax highlighting and auto linking in html_document() using the same argument as in distill (highlight_downlit = TRUE).

It won't work yet for some of the derived format like blogdown when Pandoc is not used, or bookdown because currently downlit messes up the HTML file with xml2 which breaks the weak post processing in bookdown

I'll open a new issue to track downlit auto-linking support for github_document().

[hadley]

@cderv we should also figure what needs patching to support the native pipe in pandoc's highlighter.

[cderv]

Yes this is tracked in #2196 to provide a specific syntax file to Pandoc for this. Also a patch to send upstream.
This should be done very soon.

I'll also check what downlit it doing. Not sure it does highlight specifically the operator

downlit::highlight("1:3 |> mean()", downlit::classes_pandoc())
#> [1] "<span class='fl'>1</span><span class='op'>:</span><span class='fl'>3</span> |&gt; <span class='fu'><a href='https://rdrr.io/r/base/mean.html'>mean</a></span><span class='op'>(</span><span class='op'>)</span>"

I think it should be like %>% using <span class='op'>

library(magrittr)
downlit::highlight("1:3 %>% mean()", downlit::classes_pandoc())
#> [1] "<span class='fl'>1</span><span class='op'>:</span><span class='fl'>3</span> <span class='op'>%&gt;%</span> <span class='fu'><a href='https://rdrr.io/r/base/mean.html'>mean</a></span><span class='op'>(</span><span class='op'>)</span>"

[hadley]

Can you please file an issue in downlit so I don't forget about it?

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.