Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Start-Process" fails on Windows with julia engine #10111

Open
EvoArt opened this issue Jun 24, 2024 · 13 comments · May be fixed by #11659
Open

"Start-Process" fails on Windows with julia engine #10111

EvoArt opened this issue Jun 24, 2024 · 13 comments · May be fixed by #11659
Assignees
Labels
bug Something isn't working julia windows
Milestone

Comments

@EvoArt
Copy link

EvoArt commented Jun 24, 2024

Bug description

Hi there,

I have recently started trying to use the native julia engine to render quarto docs, but so far have not managed to do so. Currently, the problem seems to be with the Start-Process function.

Any advice would be greatly appreciated, and many, many thanks for the general awesomeness of Quarto.

Steps to reproduce

I have the following test.qmd file

---
title: "test"
engine: julia
---

```{julia}
2+1
```

Then I run quarto render test.qmd

Expected behavior

I expect to see the file rendered.

Actual behavior

Instead I receive following error message:

Starting julia control server process. This might take a while...
ERROR: Start-Process : A positional parameter cannot be found that accepts argument 'System.Object[]'.     
At line:1 char:1
+ Start-Process julia -ArgumentList --startup-file=no , --project=C:\Us ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Start-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.StartProcessCommand



Stack trace:
At line:1 char:1
+ Start-Process julia -ArgumentList --startup-file=no , --project=C:\Us ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Start-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.StartProcessCommand


    at startOrReuseJuliaServer (file:///C:/Program%20Files/Quarto/bin/quarto.js:41470:23)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async getJuliaServerConnection (file:///C:/Program%20Files/Quarto/bin/quarto.js:41537:24)
    at async executeJulia (file:///C:/Program%20Files/Quarto/bin/quarto.js:41596:18)
    at async Object.execute (file:///C:/Program%20Files/Quarto/bin/quarto.js:41341:20)
    at async renderExecute (file:///C:/Program%20Files/Quarto/bin/quarto.js:78022:27)
    at async renderFileInternal (file:///C:/Program%20Files/Quarto/bin/quarto.js:78190:43)
    at async renderFiles (file:///C:/Program%20Files/Quarto/bin/quarto.js:78058:17)
    at async render (file:///C:/Program%20Files/Quarto/bin/quarto.js:82918:21)
    at async Command.actionHandler (file:///C:/Program%20Files/Quarto/bin/quarto.js:83066:32)

Your environment

IDE:

VSCode
Version: 1.88.1 (user setup)
Commit: e170252f762678dec6ca2cc69aba1570769a5d39
Date: 2024-04-10T17:41:02.734Z
Electron: 28.2.8
ElectronBuildId: 27744544
Chromium: 120.0.6099.291
Node.js: 18.18.2
V8: 12.0.267.19-electron.0
OS: Windows_NT x64 10.0.19045

OS:

Edition Windows 10 Enterprise
Version 22H2
Installed on ‎11/‎03/‎2021
OS build 19045.4529
Experience Windows Feature Experience Pack 1000.19058.1000.0

Hardware:

Device name 5CG0276F8T
Processor Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz 2.11 GHz
Installed RAM 16.0 GB (15.8 GB usable)
Device ID 44dfe63b-6a91-42b9-8fc5-0799aeec7c4d
Product ID 00330-80000-00000-AA035
System type 64-bit operating system, x64-based processor

Quarto check output

 C:\testqmd> quarto check
Quarto 1.5.47
[>] Checking versions of quarto binary dependencies...
      Pandoc version 3.2.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
      Version: 1.5.47
      Path: C:\Program Files\Quarto\bin
      CodePage: 1252

[>] Checking tools....................OK
      TinyTeX: (external install)
      Chromium: (not installed)

[>] Checking LaTeX....................OK
      Using: TinyTex
      Path: C:\Users\arn203\AppData\Roaming\TinyTeX\bin\windows\
      Version: 2024

[>] Checking basic markdown render....OK

[>] Checking Python 3 installation....OK
      Version: 3.9.2
      Path: C:/Users/arn203/AppData/Local/Programs/Python/Python39/python.exe
      Jupyter: 4.11.1
      Kernels: julia-1.6, julia-1.7, julia-1.9, julia-12-threads-1.7, julia-8-threads-1.7, julia-image-1.9, julia-omni-1.9, julia_8_threads-1.7, python3

(/) Checking Jupyter engine render....[IPKernelApp] ERROR | No such comm target registered: quarto_kernel_setup
[>] Checking Jupyter engine render....OK

[>] Checking R installation...........OK
      Version: 4.3.1
      Path: C:/PROGRA~1/R/R-43~1.1
      LibPaths:
        - C:/Users/arn203/AppData/Local/R/win-library/4.3
        - C:/Program Files/R/R-4.3.1/library
      knitr: 1.44
      rmarkdown: 2.25

[>] Checking Knitr engine render......OK
@EvoArt EvoArt added the bug Something isn't working label Jun 24, 2024
@EvoArt
Copy link
Author

EvoArt commented Jun 24, 2024

from this line in the error Start-Process julia -ArgumentList --startup-file=no , --project=C:\Us...

compared with an example from the docs at https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/start-process?view=powershell-7.4

Start-Process -FilePath "`$env:comspec" -ArgumentList "/c dir `"%SystemDrive%\Program Files`""

it looks like it's just that the ArgumentList is not enclosed in quotation marks.

@mcanouil
Copy link
Collaborator

What version of PowerShell are you running? Windows native PowerShell is not 7.4, it is 5.1.

$PSVersionTable.PSVersion

Do you have the same error, when running Quarto, for example at the root of a drive? (The issue might be triggered because of something in your file path, but cannot be sure since we don't have it.)

@EvoArt
Copy link
Author

EvoArt commented Jun 24, 2024

Thanks for looking into this. I am on version PowerShell 5.1.

I have been attempting to run this in C:\testqmd\test.qmd since I wondered if there were filepath issues.

@EvoArt
Copy link
Author

EvoArt commented Jun 24, 2024

The same example: -ArgumentList followed by a list enclosed in quotation marks is on the PowerShell 5.1 docs https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/start-process?view=powershell-5.1 (example 7).

@mcanouil mcanouil added the triaged-to Issues that were not self-assigned, signals that an issue was assigned to someone. label Jun 24, 2024
@cderv
Copy link
Collaborator

cderv commented Jun 24, 2024

The Julia Kernel is directly using Powershell and we need to adapt this command

// We need to spawn the julia server in its own process that can outlive quarto.
// Apparently, `run(detach(cmd))` in julia does not do that reliably on Windows,
// at least deno never seems to recognize that the spawning julia process has finished,
// presumably because it waits for the active child process to exit. This makes the
// tests on windows hang forever if we use the same launching mechanism as for Unix systems.
// So we utilize powershell instead which can start completely detached processes with
// the Start-Process commandlet.
if (Deno.build.os === "windows") {
const command = new Deno.Command(
"PowerShell",
{
args: [
"-Command",
"Start-Process",
options.julia_cmd,
"-ArgumentList",
// string array argument list, each element but the last must have a "," element after
"--startup-file=no",
",",
`--project=${juliaProject}`,
",",
resourcePath("julia/quartonotebookrunner.jl"),
",",
transportFile,
// end of string array
"-WindowStyle",
"Hidden",
],
env: {
"JULIA_LOAD_PATH": "@;@stdlib", // ignore the main env
},
},
);

I believed I tried the Julia notebook and did not get the problem... So maybe there is really a difference between Powershell 7 and 5 🤔

@cderv cderv removed the triaged-to Issues that were not self-assigned, signals that an issue was assigned to someone. label Jun 24, 2024
@cderv cderv added this to the v1.6 milestone Jun 24, 2024
@cderv
Copy link
Collaborator

cderv commented Jun 24, 2024

Though PowerShell is the name for the CLI for PowerShell 5 not 7. So Powershell 5 will be always used.

I need to investigate this more to really understand the problem - especially because our Windows tests seems be passing.

@devmotion
Copy link

devmotion commented Jul 19, 2024

Any update on this issue or any workaround (apart from switching to the Jupyter engine)? I just tried to rerun a very simple Quarto document with the native Julia engine on Windows that was originally written on a Linux computer, but I ran into the same problem.

@mcanouil
Copy link
Collaborator

@devmotion Everything / every update are in the thread.

@devmotion
Copy link

Thanks, I had hoped there had been already further undocumented progress since the last activity in the thread a month ago.

Based on the error message and the powershell docs, as discussed above it seems that strings in the string array for "-ArgumentList" are incorrectly not enclosed in quotation marks which causes problems e.g. with : in the --project=C:\... argument. I wonder what the value of juliaProject in https://github.com/quarto-dev/quarto-cli/blob/87abd1559c957be4d783516148c503164a2f6bcb/src/execute/julia.ts#L317C26-L317C37 is in the Windows tests on Github - maybe it's a relative path without : which could explain why the problem does not show up in the Github tests?

@cderv
Copy link
Collaborator

cderv commented Jul 22, 2024

FWIW I am on Windows, and I don't see this error. Document renders fine for me without error

I'll try to investigate closer - I am not sure why this would be working on my windows and not others if this is a quote issue.

@cderv
Copy link
Collaborator

cderv commented Jul 22, 2024

I believe this could be due with Space in the project use, or any other element in the call. I can at least reproduce with a space
powershell

> $env:QUARTO_JULIA_PROJECT="C:\Users\chris\Documents\test-dir with space"
> quarto render index.qmd --no-execute-daemon
Starting julia control server process. This might take a while...
ERROR: Start-Process : Impossible de trouver un param�tre positionnel acceptant l'argument ��with��.
Au caract�re Ligne:1 : 1
+ Start-Process julia -ArgumentList --startup-file=no , --project=C:\Us ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument : (:) [Start-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.StartProcessCommand



Stack trace:
Au caract�re Ligne:1 : 1
+ Start-Process julia -ArgumentList --startup-file=no , --project=C:\Us ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument : (:) [Start-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.StartProcessCommand


    at startOrReuseJuliaServer (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/execute/julia.ts:337:15)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async getJuliaServerConnection (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/execute/julia.ts:423:22)
    at async executeJulia (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/execute/julia.ts:516:16)
    at async Object.execute (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/execute/julia.ts:141:16)
    at async renderExecute (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/render-files.ts:232:25)
    at async renderFileInternal (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/render-files.ts:592:35)
    at async renderFiles (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/render-files.ts:325:9)
    at async renderProject (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/project.ts:441:23)
    at async Command.actionHandler (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/cmd.ts:248:26)

so definitely some quoting issue to fix. I'll target this, and we'll see if it fixes all of your errors here.

@devmotion
Copy link

#10489 seems to be a duplicate. There's a workaround described in #10489 (comment) (indeed changing the -ArgumentList) that seems to fix the problem.

@pokapokapoka
Copy link

It seems that the resourcePath("julia/quartonotebookrunner.jl") in the environment where the error occurs might include a space in the path.
In my environment, Quarto (installed via winget) is located in C:\Program Files\Quarto.

❯ (Get-Command quarto).Source
C:\Program Files\Quarto\bin\quarto.exe

Therefore, the resourcePath("julia/quartonotebookrunner.jl") becomes:
C:\Program Files\Quarto\share\julia\quartonotebookrunner.jl
which includes a space.

@cscheid cscheid modified the milestones: v1.6, v1.7 Nov 14, 2024
@jkrumbiegel jkrumbiegel linked a pull request Dec 11, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working julia windows
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants