error rendering on Windows laptop with home directory name in Chinese

[paciorek]

Bug description

I'm trying to help a student who seems to have the same problem on Windows reported in issue #4103 .
(Let me know if I should reopen that issue instead.)

We've run the commands suggested there at the end the thread by @cderv and here are the results. Any suggestions that I can I try with the student?

PS C:\Users\王茜舒> Get-ItemPropertyValue -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Nls\CodePage" -Name ACP
936
PS C:\Users\王茜舒> Get-ItemPropertyValue -Path "HKCU:\SYSTEM\CurrentControlSet\Control\Nls\CodePage" -Name ACP
Get-ItemPropertyValue : 找不到路径“HKCU:\SYSTEM\CurrentControlSet\Control\Nls\CodePage”，因为该路径不存在。
所在位置 行:1 字符: 1
+ Get-ItemPropertyValue -Path "HKCU:\SYSTEM\CurrentControlSet\Control\N ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (HKCU:\SYSTEM\Cu...ol\Nls\CodePage:String) [Get-ItemPropertyValue], Item
   NotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemPropertyValueCommand

PS C:\Users\王茜舒> Get-ChildItem -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Nls\CodePage"


    Hive: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage


Name                           Property
----                           --------
EUDCCodeRange                  932 : F040-F9FC
                               936 : AAA1-AFFE,F8A1-FEFE,A140-A7A0
                               949 : C9A1-C9FE,FEA1-FEFE
                               950 : FA40-FEFE,8E40-A0FE,8140-8DFE,C6A1-C8FE


PS C:\Users\王茜舒> Get-ChildItem -Path "HKCU:\SYSTEM\CurrentControlSet\Control\Nls\CodePage"
Get-ChildItem : 找不到路径“HKEY_CURRENT_USER\SYSTEM\CurrentControlSet\Control\Nls\CodePage”，因为该路径不存在。
所在位置 行:1 字符: 1
+ Get-ChildItem -Path "HKCU:\SYSTEM\CurrentControlSet\Control\Nls\CodeP ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (HKEY_CURRENT_US...ol\Nls\CodePage:String) [Get-ChildItem], ItemNotFound
   Exception
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand

PS C:\Users\王茜舒> Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Nls\CodePage"


10000             : c_10000.nls
10001             : c_10001.nls
10002             : c_10002.nls
10003             : c_10003.nls
10004             : c_10004.nls
10005             : c_10005.nls
10006             : c_10006.nls
10007             : c_10007.nls
10008             : c_10008.nls
10010             : c_10010.nls
10017             : c_10017.nls
10021             : c_10021.nls
10029             : c_10029.nls
10079             : c_10079.nls
10081             : c_10081.nls
10082             : c_10082.nls
1026              : c_1026.nls
1047              : c_1047.nls
1140              : c_1140.nls
1141              : c_1141.nls
1142              : c_1142.nls
1143              : c_1143.nls
1144              : c_1144.nls
1145              : c_1145.nls
1146              : c_1146.nls
1147              : c_1147.nls
1148              : c_1148.nls
1149              : c_1149.nls
1250              : c_1250.nls
1251              : c_1251.nls
1252              : c_1252.nls
1253              : c_1253.nls
1254              : c_1254.nls
1255              : c_1255.nls
1256              : c_1256.nls
1257              : c_1257.nls
1258              : c_1258.nls
1361              : c_1361.nls
20000             : c_20000.nls
20001             : c_20001.nls
20002             : c_20002.nls
20003             : c_20003.nls
20004             : c_20004.nls
20005             : c_20005.nls
20105             : c_20105.nls
20106             : c_20106.nls
20107             : c_20107.nls
20108             : c_20108.nls
20127             : c_20127.nls
20261             : c_20261.nls
20269             : c_20269.nls
20273             : c_20273.nls
20277             : c_20277.nls
20278             : c_20278.nls
20280             : c_20280.nls
20284             : c_20284.nls
20285             : c_20285.nls
20290             : c_20290.nls
20297             : c_20297.nls
20420             : c_20420.nls
20423             : c_20423.nls
20424             : c_20424.nls
20833             : c_20833.nls
20838             : c_20838.nls
20866             : c_20866.nls
20871             : c_20871.nls
20880             : c_20880.nls
20905             : c_20905.nls
20924             : c_20924.nls
20932             : c_20932.nls
20936             : c_20936.nls
20949             : c_20949.nls
21025             : c_21025.nls
21027             : c_21027.nls
21866             : c_21866.nls
28591             : C_28591.NLS
28592             : C_28592.NLS
28593             : c_28593.nls
28594             : C_28594.NLS
28595             : C_28595.NLS
28596             : C_28596.NLS
28597             : C_28597.NLS
28598             : c_28598.nls
28599             : c_28599.nls
28603             : c_28603.nls
28605             : c_28605.nls
37                : c_037.nls
38598             : c_28598.nls
437               : c_437.nls
500               : c_500.nls
50220             : c_is2022.dll
50221             : c_is2022.dll
50222             : c_is2022.dll
50225             : c_is2022.dll
50227             : c_is2022.dll
50229             : c_is2022.dll
51949             : c_20949.nls
52936             : c_is2022.dll
54936             : c_g18030.dll
55000             : c_gsm7.dll
55001             : c_gsm7.dll
55002             : c_gsm7.dll
55003             : c_gsm7.dll
55004             : c_gsm7.dll
57002             : c_iscii.dll
57003             : c_iscii.dll
57004             : c_iscii.dll
57005             : c_iscii.dll
57006             : c_iscii.dll
57007             : c_iscii.dll
57008             : c_iscii.dll
57009             : c_iscii.dll
57010             : c_iscii.dll
57011             : c_iscii.dll
708               : c_708.nls
720               : c_720.nls
737               : c_737.nls
775               : c_775.nls
850               : c_850.nls
852               : c_852.nls
855               : c_855.nls
857               : c_857.nls
858               : c_858.nls
860               : c_860.nls
861               : c_861.nls
862               : c_862.nls
863               : c_863.nls
864               : c_864.nls
865               : c_865.nls
866               : c_866.nls
869               : c_869.nls
870               : c_870.nls
874               : c_874.nls
875               : c_875.nls
932               : c_932.nls
936               : c_936.nls
949               : c_949.nls
950               : c_950.nls
AllowDeprecatedCP : 1111573537
OEMHAL            : vgaoem.fon
ACP               : 936
OEMCP             : 936
MACCP             : 10008
PSPath            : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePag
                    e
PSParentPath      : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls
PSChildName       : CodePage
PSDrive           : HKLM
PSProvider        : Microsoft.PowerShell.Core\Registry



PS C:\Users\王茜舒> Get-ItemProperty -Path "HKCU:\SYSTEM\CurrentControlSet\Control\Nls\CodePage"
Get-ItemProperty : 找不到路径“HKCU:\SYSTEM\CurrentControlSet\Control\Nls\CodePage”，因为该路径不存在。
所在位置 行:1 字符: 1
+ Get-ItemProperty -Path "HKCU:\SYSTEM\CurrentControlSet\Control\Nls\Co ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (HKCU:\SYSTEM\Cu...ol\Nls\CodePage:String) [Get-ItemProperty], ItemNotFo
   undException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemPropertyCommand

Steps to reproduce

Rendering any basic Quarto document causes the problem.

Expected behavior

One should see the rendered doc.

Actual behavior

processing file: 333.qmd
                                                                                                            
output file: [333.knit.md](http://333.knit.md/)

pandoc 
  to: html
  output-file: 333.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png
  
metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: testfile
  editor: visual
  
Error running filter D:/下载/RStudio/resources/app/bin/quarto/share/filters/main.lua:
[string "..."]:268: cannot open file 'C:\Users\??????\AppData\Local\Temp\quarto-session65e9675c\8949e2c2\39889c71' (Invalid argument)
stack traceback:
	[string "..."]:268: in function 'io.lines'
	[string "..."]:1501: in field 'processDependencies'
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:5386: in function <.../RStudio/resources/app/bin/quarto/share/filters/main.lua:5383>
	[C]: in ?
	[C]: in method 'walk'
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:171: in function 'run_emulated_filter'
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:449: in local 'callback'
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:454: in upvalue 'run_emulated_filter_chain'
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:495: in function <.../RStudio/resources/app/bin/quarto/share/filters/main.lua:476>
stack traceback:
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:171: in function 'run_emulated_filter'
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:449: in local 'callback'
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:454: in upvalue 'run_emulated_filter_chain'
	.../RStudio/resources/app/bin/quarto/share/filters/main.lua:495: in function <.../RStudio/resources/app/bin/quarto/share/filters/main.lua:476>

Your environment

• OS: Windows 11
• Rstudio: 2023.12.1-403

Quarto check output

D:\Desktop\test>quarto check
A new release of Deno is available: 1.28.2 → 1.40.2 Run `deno upgrade` to install it.

[>] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.1: OK
      Dart Sass version 1.55.0: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
      Version: 1.3.450
      Path: D:\下载\RStudio\resources\app\bin\quarto\bin
      CodePage: unknown

(|) Checking basic markdown render....Error running filter D:/下载/RStudio/resources/app/bin/quarto/share/filters/main.lua:
[string "..."]:268: cannot open file 'C:\Users\??????\AppData\Local\Temp\quarto-session3fcbaa4b\381da427\5f379e51' (Invalid argument)
stack traceback:
        [string "..."]:268: in function 'io.lines'
        [string "..."]:1501: in field 'processDependencies'
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:5386: in function <.../RStudio/resources/app/bin/quarto/share/filters/main.lua:5383>
        [C]: in ?
        [C]: in method 'walk'
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:171: in function 'run_emulated_filter'
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:449: in local 'callback'
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:454: in upvalue 'run_emulated_filter_chain'
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:495: in function <.../RStudio/resources/app/bin/quarto/share/filters/main.lua:476>
stack traceback:
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:171: in function 'run_emulated_filter'
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:449: in local 'callback'
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:454: in upvalue 'run_emulated_filter_chain'
        .../RStudio/resources/app/bin/quarto/share/filters/main.lua:495: in function <.../RStudio/resources/app/bin/quarto/share/filters/main.lua:476>
[>] Checking basic markdown render....OK

[cderv]

Thanks for the report @paciorek !

It seems the codepage use is 936 on this computer. Quarto check does return unknown though for some reason. So I don't know why this isn't read properly.

Can you update to Quarto 1.4 latest stable release and run quarto check again ?

@dragonstyle do you still have local windows 11 with codepage 936 available ? Otherwise, I'll try to set one up.

[rpbartczuk]

Hi, have very similar issue. I updated quarto and it doesn't read my codepage.
C:\Program Files\Quarto\bin>chcp Active code page: 852

C:\Program Files\Quarto\bin>quarto check
Quarto 1.4.549
[>] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.11: OK
      Dart Sass version 1.69.5: OK
      Deno version 1.37.2: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
      Version: 1.4.549
      Path: C:\Program Files\Quarto\bin
      CodePage: unknown

[>] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[>] Checking LaTeX....................OK
      Tex:  (not detected)


(|) Checking basic markdown render....Error running filter C:/Program Files/Quarto/share/filters/main.lua:
[string "..."]:267: cannot open file 'C:\Users\Rafa?\AppData\Local\Temp\quarto-sessiona3371d6a\60be6b22\128fbc86' (Invalid argument)
stack traceback:
        [string "..."]:267: in function 'io.lines'
        [string "..."]:1593: in field 'processDependencies'
        C:/Program Files/Quarto/share/filters/main.lua:7347: in field 'Meta'
        C:/Program Files/Quarto/share/filters/main.lua:240: in function 'run_emulated_filter'
        C:/Program Files/Quarto/share/filters/main.lua:936: in local 'callback'
        C:/Program Files/Quarto/share/filters/main.lua:954: in upvalue 'run_emulated_filter_chain'
        C:/Program Files/Quarto/share/filters/main.lua:990: in function <C:/Program Files/Quarto/share/filters/main.lua:987>
[>] Checking basic markdown render....OK

[cderv]

@rpbartczuk what is you username here ? C:\Users\Rafa?. I believe ? is for another character that is not correctly read ?

[eitsupi]

I saw a similar error on a Japanese version of Windows (user name contains multibyte characters).

I think they (Pandoc?) are trying to interpret a non-UTF-8 string as UTF-8 and not interpreting the path correctly.

[dragonstyle]

I am able to reproduce an error when I place the path to the quarto-cli in a directory with unicode characters (using codepage 936). I haven't yet pinned down the issue, though it appears to be a file that we are passing to pandoc that is perhaps encoded incorrectly. A basic pandoc render in the path works fine.

It's likely that the code page isn't be displayed because if there is a render exception, we clear the code page from the cache (where it is read). I'm guessing this is causing the cached code page to disappear, and perhaps the check command is using that cached value rather than computing it.

[dragonstyle]

This will reproduce in pure pandoc when the pandoc executable is placed within a unicode character path on a file system with non-english code page:

C:\Users\ct\你好>chcp
Active code page: 936

C:\Users\ct\你好>dir
 Volume in drive C has no label.
 Volume Serial Number is D466-B618

 Directory of C:\Users\ct\你好

02/20/2024  12:52 PM    <DIR>          .
02/20/2024  12:52 PM    <DIR>          ..
12/16/2023  03:20 AM       214,419,968 pandoc.exe
02/20/2024  12:47 PM                64 test.lua
02/20/2024  10:46 AM                 0 test.md

test.lua

function Pandoc(doc)
  package.path = ""
  require("foo")
end

test.md

Command

C:\Users\ct\你好>pandoc.exe test.md -L test.lua