Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-45961: [Release][Docs] Upload generated docs to GitHub Releases not apache.jfrog.io #45963

Merged
merged 1 commit into from
Apr 2, 2025

Conversation

kou
Copy link
Member

@kou kou commented Mar 28, 2025

Rationale for this change

We want to stop using apache.jfrog.io. See also: #40760

What changes are included in this PR?

Upload generated docs to GitHub Release instead of apache.jfrog.io.

Our generated docs are only for publishing. We publish them to
https://arrow.apache.org/docs/ . So we don't need any migration path
for users.

Are these changes tested?

No. I want to try this in the next release.

Are there any user-facing changes?

No.

Copy link

⚠️ GitHub issue #45961 has been automatically assigned in GitHub to PR creator.

…ses not apache.jfrog.io

We want to stop using apache.jfrog.io. See also: apache#40760

Our generated docs are only for publishing. We publish them to
https://arrow.apache.org/docs/ . So we don't need any migration path
for users.
@kou kou force-pushed the release-docs-github-release branch from a62c717 to 533973f Compare March 30, 2025 01:36
@kou kou marked this pull request as ready for review March 30, 2025 01:36
@kou
Copy link
Member Author

kou commented Mar 30, 2025

Rebased. I'll merge in a few days if nobody objects this.

@assignUser
Copy link
Member

-0
How many files will this end up being? Having the wheels and such be part of the github release makes sense, people are expecting that if they are not used to ASF ways. This on the other hand only uses the release area for temporary storage, littering it with files the users are not interested in.

I think artifacts would be a more appropriate storage for this and are easily manipulated using gh. Or should we create a workflow in arrow-site that uses the released binaries to build the docs?

@kou
Copy link
Member Author

kou commented Mar 30, 2025

How many files will this end up being?

By this PR? Or by all related PRs such as #45962 and #45964?

This PR adds 3 files: docs.tar.gz{,.asc,.sha512}

All related PRs: 180:

3 [1 [source archive] * 3 [original + .asc + .sha512]] +
3 [1 [.mltbx] * 3 [original + .asc + .sha512]] +
126 [(1 [sdist] + 41 [wheels]) * 3 [original + .asc + .sha512])] +
45 [8 [(3 [for Linux] + 4 [for macOS] + 1 [for Windows]) * 3 [original + .asc + .sha512])] +
3 [1 [docs.tar.gz] * 3 [original + .asc + .sha512]]

FYI: #45470 adds more wheels.

Having the wheels and such be part of the github release makes sense, people are expecting that if they are not used to ASF ways.

The "people" refers Python users, right? In general, Python users use PyPI not GitHub Release.

I think artifacts would be a more appropriate storage for this and are easily manipulated using gh.

If we use GitHub Actions workflow artifacts, people who verify an RC need their GitHub token to download artifacts from GitHub Actions workflow artifacts.

BTW, we need to use multiple gh commands to download from GitHub Actions wokflow artifacts (gh workflow list and gh run download). We can use one gh release download command or just curl with GitHub Release.

Or should we create a workflow in arrow-site that uses the released binaries to build the docs?

It's an option but it's a bit risky. Building docs in a workflow in arrow-site may be failed after a vote...

@pitrou
Copy link
Member

pitrou commented Mar 31, 2025

Do we actually need to upload the docs somewhere? I had no idea we were doing this. Are people downloading them?

@kou
Copy link
Member Author

kou commented Apr 1, 2025

Previously, the docs were generated after a release vote. But it sometimes failed. So, we changed to generate the docs in release process, keep it and upload it to apache/arrow-site after a release vote.

It's not for users. It's just for a stable release process.

BTW, is it strange that we use GitHub Release for artifacts for a release but not for users?

@assignUser
Copy link
Member

BTW, is it strange that we use GitHub Release for artifacts for a release but not for users?

Yes, I'd say it's a little unconventional. We have just been doing all kinds of funky stuff with it in crossbow^^

@kou
Copy link
Member Author

kou commented Apr 2, 2025

Yes, I'd say it's a little unconventional.

Hmm.

I want to simplify our release process. If we use limited places for artifacts, we can simplify our release process.

If we prefer GitHub Actions workflow artifacts to unconventional (but I don't think so...) GitHub Release usage, I'll re-implement this. But it'll spread artifacts to one more place.

We have just been doing all kinds of funky stuff with it in crossbow^^

I agree with you.

@assignUser
Copy link
Member

Oh I guess if we already have 170+ files it doesn't really matter ^^, let's test it with 20.0 I'll merge it later

@assignUser assignUser merged commit ca311c1 into apache:main Apr 2, 2025
7 checks passed
@assignUser assignUser removed the awaiting committer review Awaiting committer review label Apr 2, 2025
Copy link

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit ca311c1.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 31 possible false positives for unstable benchmarks that are known to sometimes produce them.

@kou kou deleted the release-docs-github-release branch April 3, 2025 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants