BUG: migrating from 12.6 to 12.8 breaks Python import

[spec-benno]

On Linux Ubuntu 22.08, I've recently updated the cuda-python package from 12.6 to 12.8.0 in my Python project and when I'm now trying to do:

from cuda import cuda, cudart, nvrtc

I'm getting the error message:

ImportError: cannot import name 'cuda' from 'cuda' (unknown location)

From the examples in the cuda.bindings package, I see that the same import command is still used.

Since, I'm using RDMA, I haven't tried this on Windows.

[leofang]

I cannot reproduce it (obviously, since our CI definitely got this tested), so...

• How did you install cuda-python? (pip? conda? other means?)
• Could you show the output of pip list? And if you use conda environment, also conda list?

[spec-benno]

I have an existing virtual environment with cuda-python installed and I simply did a:
pip install cuda-python==12.8

The pip list command outputs:

Package           Version
----------------- -----------
contourpy         1.3.1
cuda-bindings     12.8.0
cuda-core         0.1.1
cuda-python       12.8.0
cupy-cuda12x      13.3.0
cycler            0.12.1
fastrlock         0.8.2
flexcache         0.3
flexparser        0.4
fonttools         4.55.0
freetype-py       2.5.1
h5py              3.13.0
hsluv             5.0.4
kiwisolver        1.4.7
matplotlib        3.9.2
numpy             2.2.3
packaging         24.2
pillow            11.0.0
Pint              0.24.4
pip               22.0.2
platformdirs      4.3.6
pyparsing         3.2.0
PyQt5             5.15.11
PyQt5-Qt5         5.15.15
PyQt5_sip         12.15.0
python-dateutil   2.9.0.post0
scipy             1.14.1
setuptools        59.6.0
six               1.16.0
spcm              1.5.2
spcm_core         1.0.2
typing_extensions 4.12.2
vispy             0.14.3

[leofang]

Forgot to ask, what's the Python version?

Also, with this broken environment, could you check what happens if you do

from cuda.bindings import driver as cuda

? (Same for cudart and nvrtc)

[spec-benno]

The Python version is 3.10.12 and with the command from cuda.bindings import driver as cuda I get:

Exception has occurred: ModuleNotFoundError
No module named 'cuda.bindings'
File "...", line 13, in
from cuda.bindings import driver as cuda
ModuleNotFoundError: No module named 'cuda.bindings'

ps. small correction I'm using Ubuntu LTS 22.04

[spec-benno]

Okay, I've narrowed it down. It has to do with the migration from 12.6 to 12.8, because when setting up a new Virtual Environment with the same packages both imports work:

from cuda import cuda
from cuda.bindings import driver as cuda2

Here is the result from the pip list command in the new environment:

Package           Version
----------------- -----------
contourpy         1.3.1
cuda-bindings     12.8.0
cuda-python       12.8.0
cupy-cuda12x      13.3.0
cycler            0.12.1
fastrlock         0.8.3
flexcache         0.3
flexparser        0.4
fonttools         4.56.0
freetype-py       2.5.1
h5py              3.13.0
hsluv             5.0.4
kiwisolver        1.4.8
matplotlib        3.10.0
numpy             2.2.3
packaging         24.2
pillow            11.1.0
Pint              0.24.4
pip               25.0.1
platformdirs      4.3.6
pyparsing         3.2.1
PyQt5             5.15.11
PyQt5-Qt5         5.15.16
PyQt5_sip         12.17.0
python-dateutil   2.9.0.post0
scipy             1.15.2
six               1.17.0
spcm              1.5.2
spcm_core         1.0.2
typing_extensions 4.12.2
vispy             0.14.3

[leofang]

It seems there is a weird cleanup issue going on. With the following steps I hit an error, though different from yours

1. Create a fresh env
2. pip install "cuda-python==12.6.*"
3. pip install "cuda-python==12.8"

$ python
Python 3.12.9 | packaged by conda-forge | (main, Feb 14 2025, 08:00:06) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cuda import cuda, cudart, nvrtc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'cuda'

@spec-benno could you try a complete removal and then reinstall?

• pip uninstall -y cuda-python cuda-bindings cuda-core
• pip install cuda-python==12.8 cuda-core

and see if it is fixed?

[spec-benno]

we were thinking the same thing. Setting up a new environment does the trick, however migrating is still a problem somehow

[spec-benno]

It seems there is a weird cleanup issue going on. With the following steps I hit an error, though different from yours

1. Create a fresh env
2. pip install "cuda-python==12.6.*"
3. pip install "cuda-python==12.8"

$ python
Python 3.12.9 | packaged by conda-forge | (main, Feb 14 2025, 08:00:06) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cuda import cuda, cudart, nvrtc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'cuda'

@spec-benno could you try a complete removal and then reinstall?

• pip uninstall -y cuda-python cuda-bindings cuda-core
• pip install cuda-python==12.8 cuda-core

and see if it is fixed?

I also did the uninstall and reinstall and this works as well, for both import methods. The error you're seeing I've seen as well.

[leofang]

1. Create a fresh env
2. pip install "cuda-python==12.6.*"
3. pip install "cuda-python==12.8"

Reproducer is to change the step 2 above to pip install "cuda-python==12.6", which pip interprets as installing 12.6.0 (instead of 12.6.2.post1). Then I hit the same error:

$ python
Python 3.12.9 | packaged by conda-forge | (main, Feb 14 2025, 08:00:06) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cuda import cuda, cudart, nvrtc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'cuda' from 'cuda' (unknown location)

This seems like another unexpected issue due to layout change that caused some trouble earlier (ex: #226). I don't think there's any fix we can do but I do want to know what caused package updates dirty (since the old version is uninstalled first before the new version is installed)...