Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Port to rattler-build #1796
base: branch-25.04
Are you sure you want to change the base?
Port to rattler-build #1796
Changes from 54 commits
8ba7466
a11cbe2
93633b4
8fd3ef3
b501ca1
17f9f13
38c6feb
9b82fea
be5dbef
4994e5c
dff85ca
2d251cf
be8c03f
9303d66
84b9c14
c9e4fca
5ae6a16
00230cc
1fc7c42
e554922
5df221c
272c466
5799ee2
1dd5a9a
2b769b9
9d053ae
dd98e59
33e7351
a9067b2
c70f762
f1bf18f
fe8c6f4
2b9755b
cdb334d
d44eade
6f3b103
7ee96f4
de85796
8b2e801
a3bdd12
5489596
39856c1
f0b65fe
a05975f
d51e58a
de87ce8
f5e4669
7d17397
9c4bca8
5d0a244
13420c3
384d2d4
ea232d5
47c2e40
0a236ed
769c0a5
2aa166e
c5e8ce4
584afaa
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we actually need both of these channels or if we should tighten things up using
rapids-is-release-build
to select one or the other.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. We could probably use logic similar to https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-configure-conda-channels to define an environment variable with channels in https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-configure-rattler.
tl;dr We need to avoid using
rapidsai-nightly
ordask/label/dev
whenrapids-is-release-build
returns true.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI I opened prefix-dev/rattler-build#1424
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A final thought, should we just inline these scripts? They're pretty much all one-liners. I find that the current split makes most recipes harder to parse, not easier. Curious what other reviewers think, but I consider inlining <3 scripts preferable to having a separate file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that's a good idea and would remove one more layer of indirection
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ❤️ inline scripts too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may be different for cudf or other repos — some of the scripts are multiple lines or have a TON of flags.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd say we inline where it's a simple one-liner and if it's more complicated than that (or has a bunch of flags) we stick with the standalone install script.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whoops lol I hope at least one of you guessed that I meant "<3 line scripts" 😂
Yeah I'm fine doing this on a case-by-case basis based on the rough 3-line heuristic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably change
python/librmm/pyproject.toml
to useApache-2.0
as an SPDX identifier. https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#licenseLike this:
@vyasr Do you know why we chose to write it this way in #1529?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably fix this across the board -- I would suggest raising this with ops (in case they know of restrictions that I do not know). If ops is supportive, let's open a build-planning issue and audit this for all repos. Non-OSS repos may need a different solution, but Apache/BSD-3 repos should be fixable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then this will look like:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IIRC the current choices were made in order to guarantee compatibility with wheeltamer. @raydouglass may remember the exact list of "allowed" licenses. Given that we no longer run wheeltamer before releases that is a moot point, but I don't know if there are any subsequent scans that we have reinstated where this could still be a problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good -- that's exactly what I wondered. If we can adopt a normal SPDX license identifier in our
pyproject.toml
files, we absolutely should. I am okay with that being a follow-up to this PR, though.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, previously we used
Apache 2.0
to comply with wheeltamer. Wheeltamer does now supports the correct SPDXApache-2.0
, so would be good to switch to the SPDX identifier across RAPIDS.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK sweet then we can go ahead and synchronize the values in conda and pip so that we don't need any modification from one to the other.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Following up: I opened rapidsai/build-planning#152 to propose switching to SPDX license expressions (like
license = "Apache-2.0"
) but they're not supported by setuptools yet.To simplify things, we should update our
pyproject.toml
files tolicense = { text = "Apache-2.0" }
as we roll out rattler-build. @jakirkham proposed that here: rapidsai/build-planning#152 (comment)We should avoid using
replace(" ", "-")
in the new recipes, just fix the source when needed.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pushed up a commit adding the
-
in thepyproject.toml
files and removing thereplace
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we
load_from_file
once into something in the context and then access that data? I don't know if rattler is smart enough to cache the file on its own.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't seem to work. In all of their documentation examples they repeatedly call
load_from_file
on the same fileThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Weird. I opened prefix-dev/rattler-build#1423. This seems like a bug in the Jinja parsing.
This file was deleted.