Skip to content

PEP 796: Relative Virtual Environments [initial draft] #4476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
ceeaa3b
initial file
rickeylev Jun 27, 2025
d0a9289
add some core content
rickeylev Jun 28, 2025
41d2ea8
rename to motivation
rickeylev Jun 28, 2025
65a415c
note it enables more advanced optimizations
rickeylev Jun 28, 2025
4175b20
add foot notes, relocatable term section, links
rickeylev Jun 29, 2025
bbc85b7
update discussion, sponsor headers to pending
rickeylev Jun 29, 2025
a0c2529
rst fixups
rickeylev Jun 29, 2025
8e6e21e
fixup italics
rickeylev Jun 29, 2025
37202ca
rename to relative-venv
rickeylev Jun 29, 2025
b81c50f
clarify advanced deployment options, reflow some text
rickeylev Jun 30, 2025
55a5e28
add specification, rationale section; doc why env vars arent an option
rickeylev Jun 30, 2025
e09b00d
also mention curdir syntax handling
rickeylev Jun 30, 2025
c7a6a9b
update some todo text
rickeylev Jun 30, 2025
bc1ffbd
update codeowners
rickeylev Jun 30, 2025
2301d2a
add initial discussion footnote link
rickeylev Jun 30, 2025
e14b01f
add fr footnote
rickeylev Jun 30, 2025
175f378
revert codeowners change
rickeylev Jun 30, 2025
31cb756
remove packaging-related specs
rickeylev Jul 2, 2025
cd46f4d
fix lint errors with headers
rickeylev Jul 2, 2025
b532f5c
set sponsor
rickeylev Jul 2, 2025
99434fc
set sponsor as codeowner
rickeylev Jul 2, 2025
862142a
clarify title to focus on core pyvenv.cfg home behavior
rickeylev Jul 2, 2025
1c8fb2d
fix typo; remove host-relocatable mention; remove non-germane copying…
rickeylev Jul 2, 2025
c542890
fix typos, grammar
rickeylev Jul 2, 2025
3240227
Apply suggestions from code review
rickeylev Jul 2, 2025
557cb71
fix up markdown and text width
rickeylev Jul 2, 2025
0579623
fix title underlines
rickeylev Jul 2, 2025
c9388b3
link to relative venv reference implementation
rickeylev Jul 2, 2025
b959f24
Apply suggestions from code review
rickeylev Jul 6, 2025
5b259cc
Apply suggestions from code review
rickeylev Aug 17, 2025
560dadb
Merge branch 'main' of https://github.com/python/peps into relative.v…
rickeylev Aug 17, 2025
f41ed51
Merge branch 'relative.venvs' of https://github.com/rickeylev/peps in…
rickeylev Aug 17, 2025
5cbc9f5
better answer why/how relvenvs help; wrap some long lines
rickeylev Aug 17, 2025
0af0eea
Apply AA-Turner suggestions
rickeylev Aug 21, 2025
68dd5e0
add refs, remove CWD acronym, fix grammar
rickeylev Aug 21, 2025
b50a80c
address review comments: better split and organize rationale/motivation
rickeylev Aug 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -673,6 +673,7 @@ peps/pep-0791.rst @vstinner
peps/pep-0792.rst @dstufft
peps/pep-0793.rst @encukou
peps/pep-0794.rst @brettcannon
peps/pep-0796.rst @ncoghlan
peps/pep-0798.rst @JelleZijlstra
peps/pep-0799.rst @pablogsal
peps/pep-0800.rst @JelleZijlstra
Expand Down
322 changes: 322 additions & 0 deletions peps/pep-0796.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,322 @@
PEP: 796
Title: Relative Home Path in Virtual Environments
Author: Richard Levasseur <[email protected]>
Sponsor: Alyssa Coghlan <[email protected]>
Discussions-To: Pending
Status: Draft
Type: Standards Track
Created: 02-Jul-2025
Python-Version: 3.15


Abstract
========

This PEP formally specifies the use of a relative path for ``home``
in a Python virtual environment's :file:`pyvenv.cfg` file.

Specifically, we will discuss how such relative paths are understood
by the Python startup process, including their conversion to absolute
paths for use by the runtime.
This is a fundamental building block for virtual environments to
become more portable.

Motivation
==========

The ``home`` field in :file:`pyvenv.cfg` is used on interpreter startup to
determine the actual Python interpreter installation that is used to execute
code in that virtual environment. Currently, this path is required to be
absolute for correct virtual environment operation because the original
`PEP 405 <https://peps.python.org/pep-0405/>`__
specifying virtual environments didn't cover any specific way of processing
relative paths, their behaviour is implementation dependent. CPython releases
up to and including CPython 3.14 resolve them relative to the current process
working directory, making them too unreliable to use in practice.

The reason to support a relative path is to support portable virtual
environments, which rely on using a host-agnostic relative path to point to
``PYTHONHOME``.
A portable virtual environment is one that can be moved between
platform-compatible hosts, which is an important feature for some projects (see
"Why portable environments matter").

The reason support for a relative ``home`` path needs to be
in the interpreter itself is because locating ``PYTHONHOME`` happens
very early in the interpreter startup process, which limits the options for
customizing how it's computed. Without the ability to specify where the
supporting Python runtime files are, the interpreter can't finish startup,
so other hook points (e.g. ``site`` initialization) never trigger.

Tools that currently look to enable virtual environment portability across
machines do so either by relying on undocumented interpreter behaviour
(Bazel, omitting the ``home`` key entirely to trigger an implementation
dependent fallback to resolving via a symlinked interpreter binary on
non-Windows systems, see `gh-135773`) or by requiring a post-installation script to be executed
after the environment is placed in its target location (
`venvstacks <https://lmstudio.ai/blog/venvstacks#publishing-environment-layer-archives>`__
).

While this PEP on its own isn't sufficient to enable portable virtual
environments, it allows tools like Bazel or venvstacks to more easily prepare
constrained environments that allow for such use cases.

Why portable virtual environments matter
----------------------------------------

Portable virtual environments are important for the efficiency and
reproducibility benefits they bring from being created once and reused multiple
times later in different locations. For example, a build farm can build a
virtual environment once, cache it, and then re-use it as-is to CI jobs.


Rationale
=========

Defining semantics for a relative ``home`` path is the chosen design for the
following reasons.

First, it is a small change to interpreter startup, in particular of an
unreliable behavior that isn't specified. Currently, relative paths are
resolved to the process's current working directory, which makes them
unreliable for use in practice.

Second, for portable virtual environments, relative paths allow more
efficient, simple, and correct reproduction of virtual environments between
hosts. This is because they can be copied as-is to different locations. Some
example capabilities this allows are:

* A build farm creating (and caching) a virtual environment, which is then
served to developers (e.g. Bazel).
* Composing virtual environments together (e.g. venvstacks).
* Installing multiple arbitrary virtual environments into a container to
save disk space.
* Layering a virtual environment atop a container image for faster image
building.
* Not needing virtual environment creation tools on the host that uses a
virtual environment.
* Exact reproduction of an application's virtual environment between a
developer's host and a production host.

Third, requiring an absolute path is inherently overly proscriptive. The
interpreter itself doesn't care whether paths are relative or absolute, merely
that they point to valid locations, so users should be given the ability to use
a path of their choosing. Given a known anchor point, it's easy to transform a
relative path to an absolute path and still retain predictable and reliable
behavior that produces correct values.

Fullying designing portable virtual environments
------------------------------------------------

This PEP purposely only focuses on the interpreter startup behavior to limit
its scope. There are multiple implementations and many design questions for how
to implement portable virtual environments work (e.g. what installers should
do), but they are separate from the Python runtime initialization phase.


Specification
=============

The ``home`` value in ``pyvenv.cfg`` is permitted to use a relative path value.
These may contain parent-directory references outside of the virtual environment root
directory.
For example:

* ``subdir/whatever/bin`` (a directory within the virtual environment)
* ``./subdir/whatever/bin`` (same as above)
* ``../../../../../elsewhere/runtime/bin`` (a directory outside the virtual
environment)

Relative paths are relative to the directory containing :file:`pyvenv.cfg`.
During interpreter startup (i.e. :file:`getpath.py`), the relative path is joined to the
directory containing ``pyvenv.cfg`` to form an absolute path.
Parent-directory references (``../``) and current
directory references (``./``) are resolved syntactically (i.e. not resolving
symlinks). Symlinks are *not* resolved prior to construction of the absolute
path to ensure semantics between a relative path and absolute path remain the
same.

For example, given
``/home/user/venv/bin/pyvenv.cfg`` with
``home = ../../runtime/./bin``, the result is ``home = /home/user/runtime/bin``,
i.e. it's equivalent to using that value verbatim in ``pyvenv.cfg``.


CPython Runtime Changes
-----------------------

The CPython runtime itself *almost* already supports relative paths. The
primitives are there, so the only change needed is to define how it resolves
relative paths for ``home`` in ``pyvenv.cfg``.

Currently, relative paths resolve relative to the process's current working
directory. Because the current working directory isn't knowable in advance, it
makes relative paths today effectively impossible.

Instead, the paths should be relative to the location of the ``pyvenv.cfg``
file. This file is chosen as the anchor point because the tool that creates the
file also has to know where the Python runtime is, so can easily calculate the
correct relative path. For tools that read the ``pyvenv.cfg``, it is also easy
to simply join the directory name of where ``pyvenv.cfg`` was found with the
path in the config file. When a person reads the config file, they can do
something similar, which results in a lower cognitive burden and helps avoid
the question of "relative to what?"

This change is only a couple of lines in the startup code. Specifically, when
parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs
to be checked if it's already absolute. If not, then join it to the directory
name of the ``pyvenv.cfg`` file. The code already knows the directory and has
helpers already exist for checking if a path is absolute and joining two
paths.

A proof-of-concept of this is implemented in the author's branch,
`rickeylev/feat.relative.pyvenv.home <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__.

Backwards Compatibility
=======================

Tools that work around the absolute ``home`` key limitation the way Bazel
and venvstacks currently do (omitting the ``home`` key, or editing it after
moving the environment) will be unaffected.

While the PEP author and sponsor aren't aware of any projects that work around
the limitation by carefully controlling the current working directory used to
launch the deployed Python environments on target systems, any such projects
would be unaffected if they already ensured the working directory was set to
the folder containing ``pyvenv.cfg`` (which seems like a plausible choice,
since that is typically the root directory of the virtual environment). In the
even more unlikely case where that assumption doesn't hold, tools generating
relative virtual environment paths will typically be aware of the underlying
base runtime Python version, and hence able to update the emitted relative path
accordingly.

Security Implications
=====================

A relative path in :file:`pyvenv.cfg` may resolve differently depending on the
location of the virtual environment. This *could* point to a surprising,
potentially malicious, location.

However, this risk already exists today because a relative path isn't
_rejected_, but resolved relative to the current working directory. This PEP
just changes the anchor point to ``pyvenv.cfg`` itself.

Similarly, the same concern exists for absolute paths. The two are
fundamentally the same because they both rely on trusting whoever created
the ``pyvenv.cfg`` file, which requires having run another tool or downloaded
something from elsewhere.


How to Teach This
=================

Teaching this should be straightforward: if you use a relative path in
``pyvenv.cfg``, then it's relative to the directory containing the
``pyvenv.cfg`` file. This is simple to explain and easy to understand for
anyone that is already familiar with handling relative filesystem paths.


Reference Implementation
========================

A reference implementation is available by using the combination of:

* Python runtime from `rickeylev/feat.relative.pyvenv.home <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__
* Relative venv from `rickeylev/relvenv <https://github.com/rickeylev/relvenv>`__

And following the
`relvenv README <https://github.com/rickeylev/relvenv/blob/main/README.md>`__.

Open Issues
===========

This PEP does not specify how to create a ``pyvenv.cfg`` with a relative path,
nor how downstream tools (e.g. installers) should identify them or process
them. These questions are best addressed separately by tool owners.

References
==========

portable virtual environment
A portable virtual environment is one that can be copied from
one host to another that is platform compatible (e.g. same OS, CPU
architecture, etc), with little or no modification or post processing.

* `rules_python <https://github.com/bazel-contrib/rules_python>`__: implements
host-relocatable virtual environments.
* `rules_py <https://github.com/aspect-build/rules_py>`__: implements
host-relocatable virtual environments.
* `python-build-standalone <https://github.com/astral-sh/python-build-standalone>`__
* `venvstacks <https://pypi.org/project/venvstacks/>`__: a tool for creating
reproducible distribution artifacts from virtual environments A relocatable
Python runtime.
* `PoC for relative home in Python startup <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__
* `Python Ideas "Making venvs relocatable friendly" discussion <https://discuss.python.org/t/making-venvs-relocatable-friendly/96177>`__
* `gh-136051: relative pyvenv.cfg home <https://github.com/python/cpython/issues/136051>`__

Rejected Ideas
==============

Relative to virtual env root
----------------------------

Having the ``home`` value in ``pyvenv.cfg`` relative to the virtual
environment's root directory would work just as well, but this idea is rejected
because it requires additional effort to compute the virtual env root.

Unspecified home means to dynamically compute home
----------------------------------------------------

Today, if a ``pyvenv.cfg`` file doesn't set ``home``, the runtime will try to
dynamically compute it by checking if the current executable (which is
typically the venv's ``bin/python3`` symlink) is a symlink and, if so, use
where that points as ``PYTHONHOME``.

While currently used as a workaround by some tools, *standardising* this
behavior is undesirable for a couple reasons:

1. It presents platform-specific issues, namely with Windows. Windows does
support symlinks, but not by default, and it can require special
permissions to do so.
2. It *requires* that a symlink be used, which precludes using otherwise
equivalent mechanisms for creating an executable (e.g. a wrapper script,
hard links, etc).

In general, symlinks work best when they aren't special cased by consumers.

Using the term "relocatable"
----------------------------

Discussions pointed out that the term "relocatable" is somewhat ambiguous and
misleading for a couple reasons.

First, absolute paths make a venv arbitrarily relocatable *within* a host, but
not between hosts, so "relocatable" requires *some* qualification for
clarity.

Second, when using relative paths that point outside the venv, the venv is only
relocatable insofar as those external artifacts are also relocated. This is an
additional nuance that requires qualification of the term.

To better avoid this confusion, "relative" is chosen, which more naturally
invites the question *"Relative to what?"*.


Using PYTHONHOME at runtime to specify home
-------------------------------------------

Using the ``PYTHONHOME`` environment variable (or any environment variable) is
problematic because it's difficult to know and control when an environment
variable should or shouldn't be inherited by subprocesses. In some cases, it's
not feasible because of how layers of programs calling programs interact.

Code generally assumes that any virtual environment will be
automatically detected and activated by the presence of ``pyvenv.cfg``, so
things work better when alterations to the environment aren't a concern.


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.
Loading