Port to rattler-build #1796

gforsyth · 2025-01-27T21:47:05Z

Summary:

`recipe.yaml`

We use the multi-output cache to avoid double-compiling. The build environment compiles things, the individual outputs call cmake --install
We make use of the built-in git functions for grabbing the short-SHA (https://rattler.build/latest/experimental_features/#git-functions)
We use load_from_file to pull in metadata from the corresponding pyproject.toml (https://rattler.build/latest/experimental_features/#load_from_filefile_path)

`build_*.sh`

We use --no-build-id to allow sccache to look in a predictable place, see: https://rattler.build/latest/tips_and_tricks/#using-sccache-or-ccache-with-rattler-build
Depending on whether rapids-is-release-build, we include either rapidsai (release) or rapidsai-nightly (non-release) in the channel listing
Channels must be specified at the command-line
We remove the build_cache directory after building so it doesn't get packaged up with the other artifacts and uploaded to S3

xref: rapidsai/build-planning#47

copy-pr-bot · 2025-01-27T21:47:09Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

gforsyth · 2025-01-27T21:56:28Z

conda/recipes/librmm/meta.yaml

-        {% if cuda_major != "11" %}
-        - cuda-cudart-dev
-        {% endif %}
+        - {{ "cuda-cudart-dev" if cuda_major == "12" else "cuda-version" }}


conda-recipe-manager doesn't like != as a comparison operator

Maybe we could do not and ==s?

Suggested change

- {{ "cuda-cudart-dev" if cuda_major == "12" else "cuda-version" }}

- {{ "cudatoolkit" if not (cuda_major == "12") else "cuda-version" }}

Yep, that works!

We want to write this in a way that only mentions CUDA 11, so that the condition can be trivially deleted when adding future major version support.

Yeah that was the idea. I just goofed on the syntax. Took another go below

#1796 (comment)

gforsyth · 2025-01-27T22:39:12Z

conda/recipes/librmm/recipe.yaml

+  version: ${{ env.get("RAPIDS_PACKAGE_VERSION") }}
+  cuda_version: ${{ (env.get('RAPIDS_CUDA_VERSION') | split('.'))[:2] | join(".") }}
+  cuda_major: ${{ (env.get('RAPIDS_CUDA_VERSION') | split('.'))[0] }}
+  date_string: ${{ env.get("RAPIDS_DATE_STRING") }}


I grabbed these from rapidsai/cugraph#4551 as currently conda-recipe-manager doesn't have support for handling converting from the extended jinja2 syntax to the subset that rattler supports

conda/recipes/librmm/meta.yaml

Co-authored-by: jakirkham <[email protected]>

This reverts commit 84b9c14.

gforsyth · 2025-02-10T21:20:11Z

well, if we remove fmt from host but leave it in run, then things get copied over correctly.

still missing all the nvtx3 files and even if I use the always_include_files directive for include/nvtx3/*, nothing shows up, so something is off

🐚 colordiff nightly-librmm rattler-librmm-nohost
15,31d14
< include/nvtx3/nvToolsExtCuda.h
< include/nvtx3/nvToolsExtCudaRt.h
< include/nvtx3/nvToolsExt.h
< include/nvtx3/nvToolsExtOpenCL.h
< include/nvtx3/nvToolsExtSync.h
< include/nvtx3/nvtx3.hpp
< include/nvtx3/nvtxDetail/nvtxImplCore.h
< include/nvtx3/nvtxDetail/nvtxImplCudaRt_v3.h
< include/nvtx3/nvtxDetail/nvtxImplCuda_v3.h
< include/nvtx3/nvtxDetail/nvtxImpl.h
< include/nvtx3/nvtxDetail/nvtxImplOpenCL_v3.h
< include/nvtx3/nvtxDetail/nvtxImplSync_v3.h
< include/nvtx3/nvtxDetail/nvtxInitDecls.h
< include/nvtx3/nvtxDetail/nvtxInitDefs.h
< include/nvtx3/nvtxDetail/nvtxInit.h
< include/nvtx3/nvtxDetail/nvtxLinkOnce.h
< include/nvtx3/nvtxDetail/nvtxTypes.h
1642,1644d1624
< lib/cmake/nvtx3/nvtx3-config.cmake
< lib/cmake/nvtx3/nvtx3-config-version.cmake
< lib/cmake/nvtx3/nvtx3-targets.cmake

vyasr · 2025-02-10T21:24:39Z

well, if we remove fmt from host but leave it in run, then things get copied over correctly.

OK interesting that certainly implicates clobbering to some degree.

still missing all the nvtx3 files and even if I use the always_include_files directive for include/nvtx3/*, nothing shows up, so something is off

Maybe worth double-checking if there are any nvtx-related files in the build environment coming from other packages. Just to rule out clobbering as a possible root cause.

gforsyth · 2025-02-10T21:27:07Z

Update: From looking at their docs, it does seem like there is some intentional compatibility baked in. It's not clear how much to expect without looking at the code, though.

It does seem to respect the CBC selectors:

with $RAPIDS_CUDA_VERSION=11.8

      "variant": {
        "c_compiler_version": "11",
        "c_stdlib": "sysroot",
        "c_stdlib_version": "2.28",
        "cmake_version": ">=3.26.4,!=3.30.0",
        "cuda_compiler": "nvcc",
        "cxx_compiler_version": "11",
        "librmm": "25.04.00 cuda11_250210_5d0a2446",
        "target_platform": "linux-64"
      },

with $RAPIDS_CUDA_VERSION=12.8

      "variant": {
        "c_compiler_version": "13",
        "c_stdlib": "sysroot",
        "c_stdlib_version": "2.28",
        "cmake_version": ">=3.26.4,!=3.30.0",
        "cuda_compiler": "cuda-nvcc",
        "cxx_compiler_version": "13",
        "librmm": "25.04.00 cuda12_250210_5d0a2446",
        "target_platform": "linux-64"
      },

gforsyth · 2025-02-10T21:57:10Z

Ok, so I'm not sure what's up with my LOCAL build environment, but I just pulled down the latest .conda package and looked over the manifest and it's identical to the rmm nightly build.

I'll investigate whether my changes are even required for the fmt fix.

gforsyth · 2025-02-10T22:05:02Z

Ok, so the changes removing fmt from the host environment ARE required to make sure those headers and shared objects get included.

the nvtx stuff seems to be a me problem, but it's working fine in CI

bdice · 2025-02-10T22:25:48Z

conda/recipes/librmm/recipe.yaml

+          - "test -d \"${PREFIX}/include/rmm\""
+    about:
+      homepage: ${{ load_from_file("python/librmm/pyproject.toml").project.urls.Homepage }}
+      license: ${{ load_from_file("python/librmm/pyproject.toml").project.license.text | replace(" ", "-") }}


We should probably change python/librmm/pyproject.toml to use Apache-2.0 as an SPDX identifier. https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license

Like this:

- license = { text = "Apache 2.0" } + license = "Apache-2.0"

@vyasr Do you know why we chose to write it this way in #1529?

We should probably fix this across the board -- I would suggest raising this with ops (in case they know of restrictions that I do not know). If ops is supportive, let's open a build-planning issue and audit this for all repos. Non-OSS repos may need a different solution, but Apache/BSD-3 repos should be fixable.

Then this will look like:

Suggested change

license: ${{ load_from_file("python/librmm/pyproject.toml").project.license.text | replace(" ", "-") }}

license: ${{ load_from_file("python/librmm/pyproject.toml").project.license }}

IIRC the current choices were made in order to guarantee compatibility with wheeltamer. @raydouglass may remember the exact list of "allowed" licenses. Given that we no longer run wheeltamer before releases that is a moot point, but I don't know if there are any subsequent scans that we have reinstated where this could still be a problem.

Good -- that's exactly what I wondered. If we can adopt a normal SPDX license identifier in our pyproject.toml files, we absolutely should. I am okay with that being a follow-up to this PR, though.

Yes, previously we used Apache 2.0 to comply with wheeltamer. Wheeltamer does now supports the correct SPDX Apache-2.0, so would be good to switch to the SPDX identifier across RAPIDS.

OK sweet then we can go ahead and synchronize the values in conda and pip so that we don't need any modification from one to the other.

Following up: I opened rapidsai/build-planning#152 to propose switching to SPDX license expressions (like license = "Apache-2.0") but they're not supported by setuptools yet.

To simplify things, we should update our pyproject.toml files to license = { text = "Apache-2.0" } as we roll out rattler-build. @jakirkham proposed that here: rapidsai/build-planning#152 (comment)

We should avoid using replace(" ", "-") in the new recipes, just fix the source when needed.

Pushed up a commit adding the - in the pyproject.toml files and removing the replace

conda/recipes/rmm/recipe.yaml

bdice · 2025-02-10T22:48:30Z

conda/recipes/rmm/recipe.yaml

+  - python:
+      imports:
+        - rmm
+      pip_check: false


Thanks for the details @vyasr. Let's copy-paste this into a build-planning issue or something that we can use to plan future work. I think we do have worthwhile action items here -- passing pip check would be a very nice-to-have validation of our packaging.

bdice · 2025-02-10T22:56:49Z

@gforsyth Conflicts will need to be resolved with #1808 -- we basically need to drop spdlog / fmt dependencies and add rapids-logger.

bdice

Approving with a minor fix and one question about making all env vars required.

Follow-up work:

SPDX licenses in pyproject.toml files
Enabling pip check

conda/recipes/librmm/recipe.yaml

conda/recipes/rmm/recipe.yaml

vyasr · 2025-02-11T05:03:34Z

ci/build_cpp.sh

+                    -c rapidsai \
+                    -c rapidsai-nightly \


I wonder if we actually need both of these channels or if we should tighten things up using rapids-is-release-build to select one or the other.

Agreed. We could probably use logic similar to https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-configure-conda-channels to define an environment variable with channels in https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-configure-rattler.

tl;dr We need to avoid using rapidsai-nightly or dask/label/dev when rapids-is-release-build returns true.

vyasr · 2025-02-11T05:05:16Z

conda/recipes/librmm/recipe.yaml

+          - if: cuda_major == "11"
+            then: cuda-cudart-dev
+    about:
+      homepage: ${{ load_from_file("python/librmm/pyproject.toml").project.urls.Homepage }}


Can we load_from_file once into something in the context and then access that data? I don't know if rattler is smart enough to cache the file on its own.

Doesn't seem to work. In all of their documentation examples they repeatedly call load_from_file on the same file

Weird. I opened prefix-dev/rattler-build#1423. This seems like a bug in the Jinja parsing.

gforsyth · 2025-02-11T18:35:35Z

@jakirkham @vyasr anything else you'd like to see here?

conda/recipes/librmm/recipe.yaml

jakirkham · 2025-02-11T20:28:02Z

conda/recipes/rmm/recipe.yaml

+about:
+  homepage: ${{ load_from_file("python/rmm/pyproject.toml").project.urls.Homepage }}
+  license: ${{ load_from_file("python/rmm/pyproject.toml").project.license.text | replace(" ", "-") }}
+  summary: ${{ load_from_file("python/rmm/pyproject.toml").project.description }}


Is it possible to assign ${{ load_from_file("python/rmm/pyproject.toml").project }} to a global variable and extract each value from it?

Suggested change

about:

homepage: ${{ load_from_file("python/rmm/pyproject.toml").project.urls.Homepage }}

license: ${{ load_from_file("python/rmm/pyproject.toml").project.license.text | replace(" ", "-") }}

summary: ${{ load_from_file("python/rmm/pyproject.toml").project.description }}

about:

homepage: ${{ project.urls.Homepage }}

license: ${{ project.license.text }}

summary: ${{ project.description }}

Not currently, no. prefix-dev/rattler-build#1423

#1796 (comment)

jakirkham

Thanks Gil! 🙏

Am wondering if we can simplify the templating with a context variable

jakirkham · 2025-02-11T20:28:57Z

conda/recipes/rmm/recipe.yaml

+  cuda_major: ${{ (env.get("RAPIDS_CUDA_VERSION") | split("."))[0] }}
+  date_string: ${{ env.get("RAPIDS_DATE_STRING") }}
+  py_version: ${{ env.get("RAPIDS_PY_VERSION") }}
+  head_rev: ${{ git.head_rev(".")[:8] }}


Maybe something like this?

Suggested change

head_rev: ${{ git.head_rev(".")[:8] }}

head_rev: ${{ git.head_rev(".")[:8] }}

project: ${{ load_from_file("python/rmm/pyproject.toml").project }}

jakirkham · 2025-02-11T20:29:28Z

conda/recipes/rmm/recipe.yaml

+about:
+  homepage: ${{ load_from_file("python/rmm/pyproject.toml").project.urls.Homepage }}
+  license: ${{ load_from_file("python/rmm/pyproject.toml").project.license.text | replace(" ", "-") }}
+  summary: ${{ load_from_file("python/rmm/pyproject.toml").project.description }}


Suggested change

about:

homepage: ${{ load_from_file("python/rmm/pyproject.toml").project.urls.Homepage }}

license: ${{ load_from_file("python/rmm/pyproject.toml").project.license.text | replace(" ", "-") }}

summary: ${{ load_from_file("python/rmm/pyproject.toml").project.description }}

about:

homepage: ${{ project.urls.Homepage }}

license: ${{ project.license.text }}

summary: ${{ project.description }}

vyasr

One last small set of questions, then this LGTM!

vyasr · 2025-02-11T20:22:35Z

ci/build_cpp.sh

+if rapids-is-release-build; then
+    RAPIDS_CHANNEL=rapidsai
+else
+    RAPIDS_CHANNEL=rapidsai-nightly
+fi


Very minor nit: I foresee this propagating everywhere. Should we turn this into a GHA tool? We could even make it return an array of usable channels so that we also have a centralized place for removing the nvidia channel once we no longer need it.

Yeah, we should. I'll get that set up

PR up in rapidsai/gha-tools#143

vyasr · 2025-02-11T20:39:02Z

ci/build_cpp.sh

+# remove build_cache directory
+rm -rf "$RAPIDS_CONDA_BLD_OUTPUT_DIR"/build_cache
+


FYI I opened prefix-dev/rattler-build#1424

vyasr · 2025-02-11T20:41:26Z

conda/recipes/librmm/recipe.yaml

+      version: ${{ version }}
+    build:
+      string: cuda${{ cuda_major }}_${{ date_string }}_${{ head_rev }}
+      script: install_librmm.sh


A final thought, should we just inline these scripts? They're pretty much all one-liners. I find that the current split makes most recipes harder to parse, not easier. Curious what other reviewers think, but I consider inlining <3 scripts preferable to having a separate file.

I think that's a good idea and would remove one more layer of indirection

I consider inlining <3 scripts preferable

I ❤️ inline scripts too.

This may be different for cudf or other repos — some of the scripts are multiple lines or have a TON of flags.

I'd say we inline where it's a simple one-liner and if it's more complicated than that (or has a bunch of flags) we stick with the standalone install script.

I consider inlining <3 scripts preferable

I ❤️ inline scripts too.

Whoops lol I hope at least one of you guessed that I meant "<3 line scripts" 😂

Yeah I'm fine doing this on a case-by-case basis based on the rough 3-line heuristic.

github-actions bot added the conda label Jan 27, 2025

gforsyth force-pushed the rattler-test branch from 9a224dd to c79968f Compare January 27, 2025 21:48

gforsyth commented Jan 27, 2025

View reviewed changes

jakirkham reviewed Jan 28, 2025

View reviewed changes

conda/recipes/librmm/meta.yaml Outdated Show resolved Hide resolved

github-actions bot added the ci label Jan 29, 2025

gforsyth and others added 13 commits January 29, 2025 14:47

WIP: convert to rattler-build

8ba7466

wip: broken, but it at least tries to build

a11cbe2

restore meta.yaml with changes required for conda-recipe-manager

93633b4

Update conda/recipes/librmm/meta.yaml

8fd3ef3

Co-authored-by: jakirkham <[email protected]>

Update conda/recipes/librmm/meta.yaml

b501ca1

Co-authored-by: jakirkham <[email protected]>

builds, but compiles for each output which is bad

17f9f13

build once and cache, then define separate outputs

38c6feb

port rmm conda recipe to rattler build

9b82fea

add comment about testing environmetn

be5dbef

set GIT_* from env, fix missing torch file

4994e5c

Fix path to _torch_allocator.cpp in .gitignore

dff85ca

Disable pip check in import test

2d251cf

Swap in rattler-build for mambabuild

be8c03f

gforsyth force-pushed the rattler-test branch from 5af3309 to be8c03f Compare January 29, 2025 19:47

gforsyth marked this pull request as ready for review January 29, 2025 19:48

gforsyth requested review from a team as code owners January 29, 2025 19:48

gforsyth requested a review from KyleFromNVIDIA January 29, 2025 19:48

gforsyth added 5 commits January 29, 2025 15:20

install rattler-build

9303d66

explicitly set CMAKE_MAKE_PROGRAM

84b9c14

Revert "explicitly set CMAKE_MAKE_PROGRAM"

c9e4fca

This reverts commit 84b9c14.

add make to build

5ae6a16

add cmake to individual outputs

00230cc

bdice reviewed Feb 10, 2025

View reviewed changes

gforsyth added 2 commits February 10, 2025 17:53

simplify sccache s3 key prefix

13420c3

remove upper bound on python and remove numba

384d2d4

gforsyth added 2 commits February 10, 2025 17:57

Merge branch 'branch-25.04' into rattler-test

ea232d5

swap in rapids-logger

47c2e40

bdice approved these changes Feb 11, 2025

View reviewed changes

conda/recipes/librmm/recipe.yaml Outdated Show resolved Hide resolved

conda/recipes/librmm/recipe.yaml Outdated Show resolved Hide resolved

conda/recipes/rmm/recipe.yaml Outdated Show resolved Hide resolved

vyasr reviewed Feb 11, 2025

View reviewed changes

gforsyth added 3 commits February 11, 2025 09:17

formatting nits

0a236ed

Select rapidsai channel based on build-type

769c0a5

Merge branch 'branch-25.04' into rattler-test

2aa166e

gforsyth changed the title ~~port to rattler-build~~ Port to rattler-build Feb 11, 2025

Remove all env-var defaults

c5e8ce4

bdice approved these changes Feb 11, 2025

View reviewed changes

bdice reviewed Feb 11, 2025

View reviewed changes

conda/recipes/librmm/recipe.yaml Show resolved Hide resolved

bdice mentioned this pull request Feb 11, 2025

Use SPDX license tags in pyproject.toml rapidsai/build-planning#152

Open

jakirkham reviewed Feb 11, 2025

View reviewed changes

Change license to SPDX-allowed format, remove replace in recipe

584afaa

github-actions bot added the Python Related to RMM Python API label Feb 11, 2025

vyasr reviewed Feb 11, 2025

View reviewed changes

gforsyth mentioned this pull request Feb 11, 2025

Add rapids-rattler-channel-string tool rapidsai/gha-tools#143

Open

	- {{ "cuda-cudart-dev" if cuda_major == "12" else "cuda-version" }}
	- {{ "cudatoolkit" if not (cuda_major == "12") else "cuda-version" }}

	license: ${{ load_from_file("python/librmm/pyproject.toml").project.license.text \| replace(" ", "-") }}
	license: ${{ load_from_file("python/librmm/pyproject.toml").project.license }}

	head_rev: ${{ git.head_rev(".")[:8] }}
	head_rev: ${{ git.head_rev(".")[:8] }}
	project: ${{ load_from_file("python/rmm/pyproject.toml").project }}

		# remove build_cache directory
		rm -rf "$RAPIDS_CONDA_BLD_OUTPUT_DIR"/build_cache

Port to rattler-build #1796

Are you sure you want to change the base?

Port to rattler-build #1796

Conversation

gforsyth commented Jan 27, 2025 • edited Loading

recipe.yaml

build_*.sh

copy-pr-bot bot commented Jan 27, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gforsyth commented Feb 10, 2025

vyasr commented Feb 10, 2025

gforsyth commented Feb 10, 2025

gforsyth commented Feb 10, 2025

gforsyth commented Feb 10, 2025

bdice Feb 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdice commented Feb 10, 2025

bdice left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gforsyth commented Feb 11, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakirkham left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vyasr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gforsyth commented Jan 27, 2025 •

edited

Loading

`recipe.yaml`

`build_*.sh`

bdice Feb 10, 2025 •

edited

Loading