Add support to launching renpy from inside flatpak

[brunoais]

Currently, in linux, the code only considers situations where the program is run outside any sandbox or that it to spawning a new process outside the sandbox is the same as from inside the sandbox. This is true for snap and AppImage but not so much for flatpak.
However, flatpak provides an API (behind a permission) to work with that. It also includes a command inside the sandbox flatpak-spawn --host.

To easily identify it's running inside flatpak, it also has the environment variable container set to flatpak

Using the env var for test, it should only be editing https://github.com/furudean/vscode-renpy-warp/blob/main/src/lib/sh.ts#L144-L146 to use flatpak-spawn --host if it's set.

If you are ok with having this feature that, I'll try making this myself and submit a pull request.

[furudean]

Sounds harmless enough. Feel free to submit a PR!

[brunoais]

The needs are actually not a small footprint in your code and I no longer think it will do good.
Because the workaround is tame enough, I'm deciding to give up with this.
Instead, I'll provide the workaround which may be documented in the README, if you want.

Go to directory where renpy.py is.
Add a file which starts the editor:

echo '#!/bin/bash
command to start editor here "$@"' > _editor.sh
chmod +x _editor.sh

Edit renpy.py.
Locate:

ROOT=$(cd "$ROOT"; pwd)

Add after:

export RENPY_VSCODE="$ROOT/_editor.sh"

if [ "$container" = 'flatpak' ]; then
    exec flatpak-spawn --host \
        "--env=WARP_WS_NONCE=$WARP_WS_NONCE" "--env=WARP_ENABLED=$WARP_ENABLED" "--env=WARP_WS_PORT=$WARP_WS_PORT" "--env=RENPY_EDIT_PY=$RENPY_EDIT_PY" \
        "$0" "$@"
fi

This works nicely enough and it won't require doing any change to the code because of flatpak.

[furudean]

@brunoais Does https://github.com/furudean/vscode-renpy-warp/releases/tag/v1.14.0 help work around this issue?

[brunoais]

I will try it soon. At current pace, I'll probably try it, at most, by wednesday. I'm in the middle of a broken build and experimenting with other things which makes inserting this in between not trivial.
But I will try it.

[brunoais]

No matter if Renpy is already running or not, it always tries to start a new process.
I noticed that the new process was starting inside the sandbox but I also knew I had a workaround detection that identifies that and opens the process outside the sandbox.

However, the errors renpy was showing ate typical of trying to run renpy inside the sandbox.
After some digging, I found this:

launch.ts:192

Seems like you are overriding all environment variables and replacing them with only your own. Is that what this does? It's the only possible explanation I can think of.

[furudean]

I mean launching Ren'Py outside Visual Studio Code and then having the process connect, if that part works. I think we can probably do more to support launching from flatpack but I'm curious if that's a decent workaround at least

[furudean]

Seems like you are overriding all environment variables and replacing them with only your own. Is that what this does? It's the only possible explanation I can think of.

Are you expecting some other environment variables to be passed there? This is currently not supported but maybe something that can be added if it fits a specific purpose

[brunoais]

I mean launching Ren'Py outside Visual Studio Code and then having the process connect, if that part works.

It doesn't work. It always tries to create a new renpy process, even when one already exists.
In my opinion, process detection is a bad idea. I don't even know if it works properly in the multiple OSs. Also even then, whether you are selecting the correct process. I prefer the simplicity of the pidfile (in my POC's case, portfile).

Are you expecting some other environment variables to be passed there?

Yes. When running inside flatpak, it's expecting to receive the env var container with the value flatpak. Otherwise, it's quite hard to distinguish.

[furudean]

In my opinion, process detection is a bad idea. I don't even know if it works properly in the multiple OSs. Also even then, whether you are selecting the correct process.

This works pretty well for Mac, Linux and Windows, which are the environments I've chosen to support. Flatpacks definitely werent a part of my design and flips things a bit on its head. I would say that redesigning this mechanism isn't a super important thing for me since flatpack users are in the minority. But I want to see if I can make some adjustments to accommodate this. I don't have a good way to test these features though, unfortunately.

Side channeling discovery in line with a pidfile or similar is not a bad idea, and I forgot why I didn't go for it initially. I will review your PoC again and see if I can figure out if it fits in or not

[brunoais]

In my design there's no pid in the file but it's easy to add one if it helps. The whole design is that you read the port number from the file and then you connect to it. It's designed such that the VN is the server and the editor is the client (which is what makes most sense to the way I organize these kinds of things hierarchically).

[furudean]

Had a think about it. I want to maintain support of multiple Ren'Py instances in the same game, which would require a more bespoke method of tracking all of all the running ports (and cleaning up when they close, sometimes unexpectedly). From that perspective, it makes more sense to have one server per Visual Studio Code workspace, as it is currently. Thoughts not fully formed yet, but I'm not sure how to solve that

[brunoais]

If you prefer like that, then I'd keep the same kind of design I did but inverted.
When the VSCode server starts on an ephemeral port, it writes the port number to a file next to the game directory of all the active workspaces.
The code belonging to the rpe is then responsible to, on start, read that file and connect to that server.
If you want, each one can identify itself differently with a nonce. I'd make the current time in hours, minutes, seconds and milliseconds as the nonce, which is created on the renpy side.
I'd also make it so the client runs in a permanently loaded module instead of a module that is reloaded often.