OCPBUGS-49347: Degraded machine-config CO due to ManagedBootImages update when upgrading an OCL cluster

[djoshy]

Closes: https://issues.redhat.com/browse/OCPBUGS-49347

- What I did

• I've moved the "stamp the boot images" function call into syncMachineConfiguration. It was originally called in syncRequiredMachineConfigPools, but in cases where the master pool update(for example, OCL upgrades) took a long time and due to the ordering of the syncs, there were potential scenarios where the stamps would be never applied. By placing this into syncMachineConfiguration, this should be no longer take place.
• I've also updated the call to only explicitly check for master pools, instead of "required" pools(which has really just been master pools, but made things confusing)

- How to verify it

• This is hard to reproduce based on the bug, but OCL upgrades should be a good way to stress test this.
• Existing e2es should pass.

[openshift-ci-robot]

@djoshy: This pull request references Jira Issue OCPBUGS-49347, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.19.0) matches configured target version for branch (4.19.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Closes: https://issues.redhat.com/browse/OCPBUGS-49347

- What I did

• I've moved the "stamp the boot images" function call into syncMachineConfiguration. It was originally called in syncRequiredMachineConfigPools, but in cases where the master pool update(for example, OCL upgrades) took a long time and due to the ordering of the syncs, there were potential scenarios where the stamps would be never applied. By placing this into syncMachineConfiguration, this should be no longer take place.
• I've also updated the call to only explicitly check for master pools, instead of "required" pools(which has really just been master pools, but made things confusing)

- How to verify it

• This is hard to reproduce based on the bug, but OCL upgrades should be a good way to stress test this.
• Existing e2es should pass.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djoshy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [djoshy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[djoshy]

/test e2e-gcp-op-ocl

[openshift-ci]

@djoshy: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-azure-ovn-upgrade-out-of-change	aff2c36	link	false	/test e2e-azure-ovn-upgrade-out-of-change
ci/prow/bootstrap-unit	aff2c36	link	false	/test bootstrap-unit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[sergiordlr]

Upgrade execution:
https://qe-private-deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gs/qe-private-deck/logs/periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-4.19-upgrade-from-stable-4.19-aws-ipi-ocl-fips-f60/1886820025159913472

New upgrade execution:
https://qe-private-deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gs/qe-private-deck/logs/periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-4.19-upgrade-from-stable-4.19-aws-ipi-ocl-fips-f60/1887065122124337152

New upgrade execution:
https://qe-private-deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gs/qe-private-deck/logs/periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-4.19-upgrade-from-stable-4.19-aws-ipi-ocl-fips-f60/1887464170945777664