Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add jobs flag to spec on pgupgrade to determine number of pgupgrade jobs #4039

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

glynn-scott-hofbauer
Copy link

Checklist:

  • Have you added an explanation of what your changes do and why you'd like them to be included?
  • Have you updated or added documentation for the change, as applicable?
  • Have you tested your changes on all related environments with successful results, as applicable?
    • Have you added automated tests?

Type of Changes:

  • New feature
  • Bug fix
  • Documentation
  • Testing enhancement
  • Other

What is the current behavior (link to any open issues here)?

Currently, the PGUpgrade job uses the default number of parallel jobs to execute the upgrade process. However, we have encountered issues on NFS filesystems where running multiple jobs simultaneously causes filesystem conflicts, leading to upgrade failures. One such error is shown below:

If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.Performing Upgrade
------------------
Setting locale and encoding for new cluster                   ok
Analyzing all rows in the new cluster                         ok
Freezing all rows in the new cluster                          ok
pg_upgrade: warning: could not remove directory "/pgdata/pg16/pg_xact": Directory not empty
Deleting files from new pg_xact                               
could not delete directory "/pgdata/pg16/pg_xact" 

To address this issue, we tested modifying the PGUpgrade job by overriding the default behavior and setting the number of jobs to 1. This adjustment resolved the filesystem conflicts, and the upgrade completed successfully.

What is the new behavior (if this is a feature change)?

  • Breaking change (fix or feature that would cause existing functionality to change)

Other Information:

@glynn-scott-hofbauer glynn-scott-hofbauer changed the title Add jobs flag to spec on pgupgrade to determine number of jobs pgupgrade Add jobs flag to spec on pgupgrade to determine number of pgupgrade jobs Nov 26, 2024
@cbandy
Copy link
Member

cbandy commented Dec 5, 2024

@glynn-scott-hofbauer thank you for this.

I took a different approach in #4043 that keeps the API the same. There, --jobs=1 for PGUpgrades with low or no CPU resources specified. Behind a new feature gate, the operator will choose a larger number of --jobs when sufficient CPU is available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants