Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to new github pages build system #416

Closed
samccann opened this issue Jun 3, 2024 · 23 comments
Closed

Update to new github pages build system #416

samccann opened this issue Jun 3, 2024 · 23 comments
Assignees

Comments

@samccann
Copy link
Contributor

samccann commented Jun 3, 2024

ansible-community/nightly-builds apparently still uses the legacy GitHub Pages build system
Need to be moved to the new system https://github.blog/2022-08-10-github-pages-now-uses-actions-by-default/

there's an example of doing this in ansible/pylibssh (at the bottom of ci-cd.yml)

@samccann samccann transferred this issue from ansible-community/nightly-builds Jun 3, 2024
@samccann samccann self-assigned this Jun 20, 2024
@samccann
Copy link
Contributor Author

Okay I'm getting myself confused here on how to proceed so could use some help.
NOTE- This weekly build/publish will fail on the next scheduled build, according to GitHub's recent email that says legacy gh-pages will stop working on June 30th.

So on to where I'm confused. Based on the GitHub docs, all I need to do in https://github.com/ansible-community/nightly-builds is enable GitHub actions. But this blog post suggests there's more work to do than just enable GH actions in the repo.

And the email from Github says:
image

And I see no warning on the Pages setting in https://github.com/ansible-community/nightly-builds BUT GH Actions are disabled in that repo.

And finally, the actual workflow that seems to update these pages comes from the antsibull repo (where it's using someone else's github actions).

So I'm inclined to enable GHA in the nightly-builds repo, manually trigger the antsibull workflow, and see what happens.
But with no warning on the Pages settings, I'm unsure if I'm fixing anything with that step or not.

Finally, I do think we shoud start using the GHA for pages that are from github, not that third party one.

@samccann
Copy link
Contributor Author

@felixfontein @webknjaz what do you think about ^^? I'll turn on GHA later today in the nightly-build repo to see what happens but would like your thoughts on if I need to do more.

@felixfontein
Copy link
Contributor

My guess is that the repo is populated by GA from https://github.com/ansible-community/antsibull/blob/main/.github/workflows/build-dumb-pypi.yml, so enabling GHA in https://github.com/ansible-community/nightly-builds doesn't make sense.

@felixfontein
Copy link
Contributor

Hmm, we're also using publishing from a branch for the collection docs publishing, for example https://ansible-collections.github.io/community.docker/branch/main/. CC @briantist - all these will probably also stop working on June 30th then.

I'll do some tests with this tomorrow.

@felixfontein
Copy link
Contributor

(I'm somewhat surprised that we didn't get this warning for all the other repositories in github.com/ansible-collections that use the legacy way of publishing docs as well.)

@samccann
Copy link
Contributor Author

yes, I'm finding it difficult to see how to replace the old third-party GHA (https://github.com/ansible-community/antsibull/blob/main/.github/workflows/build-dumb-pypi.yml#L193) with the new GHA from Github. The old one allows you to push to an external repo (nightly builds) but the new one doesn't.

@samccann
Copy link
Contributor Author

Actions are enabled for community.docker - https://github.com/ansible-collections/community.docker/settings/actions
(and my guess - for all the other collections). The Github docs say that's all you have to change. For nightly-builds - it's disabled (and thus that's the one giving us warning email(s). Though the warning email also said the Pages under settings should have a warning there as well and it doesn't

@felixfontein
Copy link
Contributor

I don't think so; you also need to change the source for the built pages, and install a workflow for publishing (if it's not already there).

For the nightly repo, that workflow must not be in the gh-pages branch but somewhere else, otherwise it is removed every time the data is published.

@samccann
Copy link
Contributor Author

So for community.docker - it is using the 'new way '. Notice in the actions it has https://github.com/ansible-collections/community.docker/actions/workflows/pages/pages-build-deployment now
That's what is doing the Githhub GHA to publish to pages.

@samccann
Copy link
Contributor Author

samccann commented Jun 26, 2024

Sorry @felixfontein - not understanding your prior comment?

@felixfontein
Copy link
Contributor

So for community.docker - it is using the 'new way '.

It is not. It is simply storing the raw pages on the gh-pages branch. That here:

Notice in the actions it has https://github.com/ansible-collections/community.docker/actions/workflows/pages/pages-build-deployment now

Is basically the legacy GitHub Pages build system. (It doesn't exist as a workflow in the repo.)

@felixfontein
Copy link
Contributor

felixfontein commented Jun 26, 2024

(In case you looked into the settings: I changed it to "source: GHA" for that repo to see what actually happens then; I didn't intend to actually change it now, but it seemed to have auto-changed it. It's now back to its original value: "Deploy from a branch")

@samccann
Copy link
Contributor Author

This is what GitHub emailed

Hello,

We are writing to inform you about an important update regarding GitHub Pages that could impact one or multiple sites you own.

What is happening?

Delivering on our continued commitment to enhance security, protect your data, and deliver an overall improved experience, we will be shutting down our legacy pagesworker architecture by June 30th 2024.

What is replacing legacy pagesworker?

Since August 2022, GitHub Pages sites have been built and deployed with GitHub Actions. However for repositories where Actions is unavailable, we used the legacy pagesworkers for Pages builds and deployments.

Going forward, GitHub Actions will become a requirement for deploying Pages sites that require a build step, such as those utilizing Jekyll. We chose to implement this change because GitHub Actions provides developers a modern, secure sandbox that surpasses the capabilities of pagesworker. With GitHub Actions, you get your choice of operating system, log streaming, dependency caching, custom gates, and modern CI/CD features.

Additionally, we are pleased to announce the general availability of the feature allowing you to delegate Pages deployments to GitHub Actions alongside a source branch.

How can I tell if my Pages site will be impacted?

If your Pages site is being built by the legacy infrastructure, a warning message will be displayed in your Pages settings starting today. Note that the vast majority of pages sites have already been leveraging GitHub Actions. We will only fall back to the legacy build infrastructure if GitHub Actions is unavailable in your repository.

Transitioning to Actions

Transitioning your Pages builds from pagesworker to Actions is straightforward. Simply enable Actions in your repository settings to ensure your site continues to build.

For those on legacy paid plans (Gold, Silver, etc.) where Actions is unavailable, we suggest the following:

For Public repositories, consider transferring it to another organization where Actions is available.
For Private repositories with limited contributors, consider transferring the repository to another organization with a current per-seat billing plan. If billing is a concern, consider using a third-party CI solution to push static content directly to your publishing branch. Adding a .nojekyll file will allow your site to deploy without Actions.

@webknjaz
Copy link
Member

FWIW I still think the pages should be moved into the build-data repo or antsibull. The reason is that it'll get more complicated to trigger stuff cross-repos. One of the original workflow intentions was to catch regressions in the packaging automation there. It's not just about serving the files as a nightly package index, I remember it did catch problems with the automation in the past.

@felixfontein
Copy link
Contributor

I'm not sure it's a good idea to move it there, since the files in that repo take up quite some space (especially if old commits are kept around / still referenced somehow), so this could accidentally blow up these repos with temp files that really don't belong in them.

@webknjaz
Copy link
Member

@felixfontein with the GHA-based GH Pages publishing it won't use the Git storage/branches, it'll create a GHA workflow artifact and then publish it. I suppose we could also auto-delete it (I don't remember if GH does that automatically).

@felixfontein
Copy link
Contributor

I still don't think this should be added to either the antsibull or the ansible-build-data repos.

with the GHA-based GH Pages publishing it won't use the Git storage/branches,

That depends on how it is set up. The way GitHub recommends in the blog post, it uses a branch from the repo. I guess it can now be done without that, but I think it should still be done in a separate repo.

@webknjaz
Copy link
Member

Just checked https://github.com/ansible/pylibssh/actions/runs/9465600458#artifacts and I see that all the workflow artifacts are expired, which reminded me of the retention setting. So I've gone to the action and confirmed that it's just one day: https://github.com/actions/upload-pages-artifact/blob/1780dfc/action.yml#L16.

Interestingly, the action README talks about 10 GB being the maximum size. I remember the old-style GH Pages being kept under 500 MB.

@webknjaz
Copy link
Member

The way GitHub recommends in the blog post, it uses a branch from the repo.

That is the old-style mechanism that's being phased out. The new thing only needs a GHA workflow and everything's generated inside. I've been using it in 3-4 repos over the past few years.

@felixfontein
Copy link
Contributor

ansible-community/github-docs-build#92 describes a solution for the docs publishing workflows. I'll look at the nightly builds tomorrow.

@felixfontein
Copy link
Contributor

I pushed a modified version of the workflow to https://github.com/ansible-community/nightly-builds/blob/main/.github/workflows/build-dumb-pypi.yml which should publish to GH Pages using the new mechanism (see the last few steps of the workflow), and I've created ansible-community/antsibull-build#608 to remove the workflow from the antsibull repository.

@felixfontein
Copy link
Contributor

Closing since this has been resolved. If you want to improve the workflow, feel free to create PRs :)

@samccann
Copy link
Contributor Author

Thanks @felixfontein !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants