Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Transition build system of cuda_cccl to scikit-build-core #3597

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Transition build system of cuda_cccl to scikit-build-core #3597

wants to merge 6 commits into from
+134 −61

Conversation

oleksandr-pavlyk
Copy link

Description

closes gh-2334

Change build system of internal Python package cuda_cccl that installs CCCL headers into Python prefix.

The advantage of using scikit-build-core is that installation of headers is controlled by project's top-level "CMakeLists.txt"

A minimal pytest-based test suite is added as well.

Checklist

  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@oleksandr-pavlyk
Rewrite of cuda_cccl build system
f3fbf3a 
Transition from using setuptools to using scikit-build-core, so
that CMakeLists.txt is used to ensure that include is the same
as in the cuda toolkit.

Added test folder to test that the package works correctly.
@oleksandr-pavlyk
Make project.version metadata dynamic
c000604 
Use suggestion by Bradley to use scikit-build-core regex facility
to extract version value from "_version.py"
@oleksandr-pavlyk
Delete existing cuda_cccl
b2be856
@oleksandr-pavlyk
Rename new_cuda_cccl to cuda_cccl to become ready to open a PR
dd191f7
@oleksandr-pavlyk oleksandr-pavlyk requested review from a team as code owners January 30, 2025 00:42
@github-actions GitHub Actions
Copy link
Contributor

🟥 CI finished in 4m 36s: Pass: 0%/1 | Total: 4m 36s | Avg: 4m 36s | Max: 4m 36s

  • 🟥 python: Pass: 0%/1 | Total: 4m 36s | Avg: 4m 36s | Max: 4m 36s

     
    🟥 cpu
  🟥 amd64              Pass:   0%/1   | Total:  4m 36s | Avg:  4m 36s | Max:  4m 36s
🟥 ctk
  🟥 12.6               Pass:   0%/1   | Total:  4m 36s | Avg:  4m 36s | Max:  4m 36s
🟥 cudacxx
  🟥 nvcc12.6           Pass:   0%/1   | Total:  4m 36s | Avg:  4m 36s | Max:  4m 36s
🟥 cudacxx_family
  🟥 nvcc               Pass:   0%/1   | Total:  4m 36s | Avg:  4m 36s | Max:  4m 36s
🟥 cxx
  🟥 GCC13              Pass:   0%/1   | Total:  4m 36s | Avg:  4m 36s | Max:  4m 36s
🟥 cxx_family
  🟥 GCC                Pass:   0%/1   | Total:  4m 36s | Avg:  4m 36s | Max:  4m 36s
🟥 gpu
  🟥 v100               Pass:   0%/1   | Total:  4m 36s | Avg:  4m 36s | Max:  4m 36s
🟥 jobs
  🟥 Test               Pass:   0%/1   | Total:  4m 36s | Avg:  4m 36s | Max:  4m 36s

👃 Inspect Changes

Modifications in project?

Project
CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
+/- python
CCCL C Parallel Library
Catch2Helper

Modifications in project or dependencies?

Project
CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
+/- python
CCCL C Parallel Library
Catch2Helper

🏃‍ Runner counts (total jobs: 1)
# Runner
1 linux-amd64-gpu-v100-latest-1

@leofang
Copy link
Member

leofang commented Jan 30, 2025

cc @bdice @vyasr for vis

@github-actions GitHub Actions
Copy link
Contributor

🟥 CI finished in 6m 03s: Pass: 0%/1 | Total: 6m 03s | Avg: 6m 03s | Max: 6m 03s

  • 🟥 python: Pass: 0%/1 | Total: 6m 03s | Avg: 6m 03s | Max: 6m 03s

     
    🟥 cpu
  🟥 amd64              Pass:   0%/1   | Total:  6m 03s | Avg:  6m 03s | Max:  6m 03s
🟥 ctk
  🟥 12.6               Pass:   0%/1   | Total:  6m 03s | Avg:  6m 03s | Max:  6m 03s
🟥 cudacxx
  🟥 nvcc12.6           Pass:   0%/1   | Total:  6m 03s | Avg:  6m 03s | Max:  6m 03s
🟥 cudacxx_family
  🟥 nvcc               Pass:   0%/1   | Total:  6m 03s | Avg:  6m 03s | Max:  6m 03s
🟥 cxx
  🟥 GCC13              Pass:   0%/1   | Total:  6m 03s | Avg:  6m 03s | Max:  6m 03s
🟥 cxx_family
  🟥 GCC                Pass:   0%/1   | Total:  6m 03s | Avg:  6m 03s | Max:  6m 03s
🟥 gpu
  🟥 v100               Pass:   0%/1   | Total:  6m 03s | Avg:  6m 03s | Max:  6m 03s
🟥 jobs
  🟥 Test               Pass:   0%/1   | Total:  6m 03s | Avg:  6m 03s | Max:  6m 03s

👃 Inspect Changes

Modifications in project?

Project
CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
+/- python
CCCL C Parallel Library
Catch2Helper

Modifications in project or dependencies?

Project
CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
+/- python
CCCL C Parallel Library
Catch2Helper

🏃‍ Runner counts (total jobs: 1)
# Runner
1 linux-amd64-gpu-v100-latest-1

@github-actions GitHub Actions
Copy link
Contributor

🟩 CI finished in 47m 48s: Pass: 100%/1 | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s

  • 🟩 python: Pass: 100%/1 | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s

     
    🟩 cpu
  🟩 amd64              Pass: 100%/1   | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s
🟩 ctk
  🟩 12.6               Pass: 100%/1   | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s
🟩 cudacxx
  🟩 nvcc12.6           Pass: 100%/1   | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/1   | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s
🟩 cxx
  🟩 GCC13              Pass: 100%/1   | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s
🟩 cxx_family
  🟩 GCC                Pass: 100%/1   | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s
🟩 gpu
  🟩 v100               Pass: 100%/1   | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s
🟩 jobs
  🟩 Test               Pass: 100%/1   | Total: 47m 48s | Avg: 47m 48s | Max: 47m 48s

👃 Inspect Changes

Modifications in project?

Project
CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
+/- python
CCCL C Parallel Library
Catch2Helper

Modifications in project or dependencies?

Project
CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
+/- python
CCCL C Parallel Library
Catch2Helper

🏃‍ Runner counts (total jobs: 1)
# Runner
1 linux-amd64-gpu-v100-latest-1

@oleksandr-pavlyk
Expand tests for cuda_cccl
792fd5d 
1. Use pytest fixture to reuse computed inc_paths
2. Split long test into smaller individual tests with descriptive names
3. Add tests to check that directories contain expected marker files (versions)
4. Avoid using c4l in favor of cccl
@github-actions GitHub Actions
Copy link
Contributor

🟩 CI finished in 45m 37s: Pass: 100%/1 | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s

  • 🟩 python: Pass: 100%/1 | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s

     
    🟩 cpu
  🟩 amd64              Pass: 100%/1   | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s
🟩 ctk
  🟩 12.6               Pass: 100%/1   | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s
🟩 cudacxx
  🟩 nvcc12.6           Pass: 100%/1   | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/1   | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s
🟩 cxx
  🟩 GCC13              Pass: 100%/1   | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s
🟩 cxx_family
  🟩 GCC                Pass: 100%/1   | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s
🟩 gpu
  🟩 v100               Pass: 100%/1   | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s
🟩 jobs
  🟩 Test               Pass: 100%/1   | Total: 45m 37s | Avg: 45m 37s | Max: 45m 37s

👃 Inspect Changes

Modifications in project?

Project
CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
+/- python
CCCL C Parallel Library
Catch2Helper

Modifications in project or dependencies?

Project
CCCL Infrastructure
libcu++
CUB
Thrust
CUDA Experimental
+/- python
CCCL C Parallel Library
Catch2Helper

🏃‍ Runner counts (total jobs: 1)
# Runner
1 linux-amd64-gpu-v100-latest-1

@leofang
Copy link
Member

leofang commented Jan 31, 2025

Q: How much efforts is needed to also build cuda.parallel with skbuild-core? I think it was the main mission of #2334 because it contains the C library to be built with nvcc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bdice bdice bdice left review comments

@shwina shwina shwina left review comments

@miscco miscco Awaiting requested review from miscco miscco is a code owner automatically assigned from NVIDIA/cccl-cmake-codeowners

@gevtushenko gevtushenko Awaiting requested review from gevtushenko gevtushenko is a code owner automatically assigned from NVIDIA/cccl-python-codeowners

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
CCCL
Status: In Review
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[ENH]: Improve/Rewrite cuda.parallel's build system
4 participants
@oleksandr-pavlyk @leofang @shwina @bdice