Skip to content

Commit

Permalink
Add instructions for using udev to manage GPU access (#325)
Browse files Browse the repository at this point in the history
* Added instructions for using udev to manage GPU access

* editing

* additional edits

* update code block syntax

* Updated wordlist to resolve linting error

* remove duplicate word from .wordlist.txt

* spell check does not ignore case :)

* Removed "recommended" from the udev section header, moved group membership to be the first method listed

* Group Membership -> group membership

Co-authored-by: Young Hui - AMD <[email protected]>

* udev Rules -> udev rules

(sorry, old habits die hard))

Co-authored-by: Young Hui - AMD <[email protected]>

* Udev -> udev rewording

Co-authored-by: Young Hui - AMD <[email protected]>

* Update .wordlist.txt

Co-authored-by: Young Hui - AMD <[email protected]>

* Update .wordlist.txt

troubleshooting linting failure

---------

Co-authored-by: Young Hui - AMD <[email protected]>
  • Loading branch information
AMD-melliott and yhuiYH authored Nov 18, 2024
1 parent fe89acc commit 76ac43a
Show file tree
Hide file tree
Showing 2 changed files with 87 additions and 11 deletions.
3 changes: 3 additions & 0 deletions .wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ CTest
DHCP
dkms
dmesg
DRM
fw
FW
FWDEV
Expand All @@ -32,3 +33,5 @@ vCPU
vfio
VFIO
vhost
udev
Udev
95 changes: 84 additions & 11 deletions docs/install/prerequisites.rst
Original file line number Diff line number Diff line change
Expand Up @@ -192,35 +192,108 @@ To install for the currently active kernel run the command corresponding to your
.. _group_permissions:

Setting permissions for groups
Configuring permissions for GPU access
================================================================

This section provides steps to add any current user to a video group to access GPU resources. We
recommend using the video group for all ROCm-supported operating systems.
There are two primary methods to configure GPU access for ROCm: group membership or
udev rules. Each method has its own advantages, and the choice depends on your
specific requirements and system management preferences.

Using group membership
--------------------------------------------------------------------

By default, GPU access is managed through membership in the ``video`` and ``render`` groups.
The ``video`` and ``render`` groups are system groups in Linux used to manage access
to graphics hardware and related functionality. Traditionally, the ``video`` group is used
to control access to video devices, including graphics cards and video capture devices.
The ``render`` group is more recent and specifically controls access to GPU rendering capabilities
through Direct Rendering Manager (DRM) render nodes.

1. To check the groups in your system, issue the following command:

.. code-block:: shell
groups
2. Add yourself to the ``render`` and ``video`` group using the command:
2. Add yourself to the ``video`` and ``render`` groups:

.. code-block:: shell
sudo usermod -a -G video,render $LOGNAME
3. Optionally, add other users to the ``video`` and ``render`` groups:

.. code-block:: shell
sudo usermod -a -G video,render user1
sudo usermod -a -G video,render user2
4. To add all future users to the render and video groups by default, run the following commands:

.. code-block:: shell
echo 'ADD_EXTRA_GROUPS=1' | sudo tee -a /etc/adduser.conf
echo 'EXTRA_GROUPS=video' | sudo tee -a /etc/adduser.conf
echo 'EXTRA_GROUPS=render' | sudo tee -a /etc/adduser.conf
Using udev rules
--------------------------------------------------------------------
A flexible way to manage device permissions is to use udev rules. They apply system-wide, can be
easily deployed via configuration management tools, and eliminate the need for user group management.
This method provides more granular control over GPU access.

Grant GPU access to all users on the system
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

1. Create a new file ``/etc/udev/rules.d/70-amdgpu.rules`` with the following content:

.. code-block:: shell
KERNEL=="kfd", MODE="0666"
SUBSYSTEM=="drm", KERNEL=="renderD*", MODE="0666"
2. Reload the udev rules:

.. code-block:: shell
sudo udevadm control --reload-rules && sudo udevadm trigger
This configuration grants all users read and write access to AMD GPU resources,
including the AMD Kernel Fusion Driver (KFD) and Direct Rendering Manager (DRM) devices.

Grant GPU access to a custom group
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

1. Create a new group (e.g., ``devteam``):

.. code-block:: shell
sudo usermod -a -G render,video $LOGNAME
sudo groupadd devteam
To add all future users to the ``render`` and ``video`` groups by default, run the following commands:
2. Add users to the new group:

.. code-block:: shell
echo 'ADD_EXTRA_GROUPS=1' | sudo tee -a /etc/adduser.conf
echo 'EXTRA_GROUPS=video' | sudo tee -a /etc/adduser.conf
echo 'EXTRA_GROUPS=render' | sudo tee -a /etc/adduser.conf
sudo usermod -a -G devteam dev1
sudo usermod -a -G devteam dev2
3. Create udev rules to assign GPU devices to this group:

Create a file ``/etc/udev/rules.d/70-amdgpu.rules`` with:

.. code-block:: shell
KERNEL=="kfd", GROUP="devteam", MODE="0660"
SUBSYSTEM=="drm", KERNEL=="renderD*", GROUP="devteam", MODE="0660"
4. Reload the udev rules:

.. code-block:: shell
.. tip::
sudo udevadm control --reload-rules && sudo udevadm trigger
On systems with multiple users, if ROCm is installed system wide, each individual user should be added to the ``render`` and ``video`` groups.
This configuration grants all users in the ``devteam`` group read and write access to AMD GPU resources,
including the Kernel Fusion Driver (KFD) and Direct Rendering Manager (DRM) devices.

Disable integrated graphics (IGP), if applicable
================================================================
Expand Down

0 comments on commit 76ac43a

Please sign in to comment.