Skip to content

OCPBUGS-54412: Do not retry extension-sourced tests #29867

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

OCPBUGS-54412: Do not retry extension-sourced tests #29867

wants to merge 1 commit into from
+11 −1

Conversation

stbenjam
Copy link
Member

@stbenjam stbenjam commented May 28, 2025 •
edited
Loading

I accidentally broke retries for extension tests, because the pointer to a test binary was omitted. The retry errors with:

error: no test exists with that name: [sig-cli] Kubectl Port forwarding Shutdown client connection while the remote stream is writing data to the port-forward connection port-forward should keep working after detect broken connection [Suite:openshift/conformance/parallel] [Suite:k8s]

However, because our end goal is that we don't retry at all, and we've existed in this state for months, I'm leaving retries off for externally sourced tests.

origin-sourced tests will continue to retry with the hope we ratchet up test quality to also stop retrying them.

@openshift-ci-robot openshift-ci-robot added the jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. label May 28, 2025
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented May 28, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 28, 2025
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 28, 2025
@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label May 28, 2025
@openshift-ci-robot
Copy link

@stbenjam: This pull request references Jira Issue OCPBUGS-54412, which is invalid:

  • expected the bug to target the "4.20.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

I accidentally broke retries for extension tests, because the pointer to a test binary was omitted. However, because our end goal is that we don't retry at all, and we've existed in this state for months, I'm leaving retries off for externally sourced tests.

origin-source tests will continue to retry with the hope we ratchet up test quality to also stop retrying them.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 28, 2025
@stbenjam stbenjam changed the title OCPBUGS-54412: Fix retry logic for extension tests OCPBUGS-54412: Do not retry extension-sourced tests May 28, 2025
@deads2k
Copy link
Contributor

deads2k commented May 28, 2025

Best bug this year!

I'm leaving retries off for externally sourced tests.

Good call.

@stbenjam
Copy link
Member Author

/test e2e-aws-ovn

@stbenjam
Copy link
Member Author

/test e2e-aws-ovn

@stbenjam
Copy link
Member Author

origin retries still work; kube tests don't: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/29867/pull-ci-openshift-origin-main-e2e-aws-ovn/1928078241470353408

@stbenjam
OCPBUGS-54412: Fix retry logic for extension tests
00ca1c5 
I accidentally broke retries for extension tests, because the pointer to
a test binary was omitted.  However, because our end goal is that we
don't retry at all, and we've existed in this state for months, I'm
leaving retries off for externally sourced tests.

origin-source tests will continue to retry with the hope we ratchet up
test quality to also stop retrying them.
@stbenjam stbenjam marked this pull request as ready for review May 29, 2025 17:03
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 29, 2025
@openshift-ci openshift-ci bot requested review from deads2k and sjenning May 29, 2025 17:05
@stbenjam
Copy link
Member Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels May 29, 2025
@openshift-ci-robot
Copy link

@stbenjam: This pull request references Jira Issue OCPBUGS-54412, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.0) matches configured target version for branch (4.20.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@sosiouxme
Copy link
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 29, 2025
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented May 29, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sosiouxme, stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 88edc2f and 2 for PR HEAD 00ca1c5 in total

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 88edc2f and 2 for PR HEAD 00ca1c5 in total

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD e47a79b and 1 for PR HEAD 00ca1c5 in total

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented May 30, 2025

@stbenjam: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-vsphere-ovn-etcd-scaling 00ca1c5 link false /test e2e-vsphere-ovn-etcd-scaling
ci/prow/e2e-aws-disruptive 00ca1c5 link false /test e2e-aws-disruptive
ci/prow/e2e-aws-ovn-serial-publicnet-1of2 00ca1c5 link false /test e2e-aws-ovn-serial-publicnet-1of2
ci/prow/e2e-gcp-fips-serial-1of2 00ca1c5 link false /test e2e-gcp-fips-serial-1of2
ci/prow/e2e-openstack-ovn 00ca1c5 link false /test e2e-openstack-ovn
ci/prow/4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback 00ca1c5 link false /test 4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback
ci/prow/e2e-aws-ovn-serial-publicnet-2of2 00ca1c5 link false /test e2e-aws-ovn-serial-publicnet-2of2
ci/prow/e2e-metal-ipi-serial-ovn-ipv6-2of2 00ca1c5 link false /test e2e-metal-ipi-serial-ovn-ipv6-2of2
ci/prow/e2e-gcp-fips-serial-2of2 00ca1c5 link false /test e2e-gcp-fips-serial-2of2
ci/prow/e2e-aws-ovn-single-node-upgrade 00ca1c5 link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/e2e-azure-ovn-upgrade 00ca1c5 link false /test e2e-azure-ovn-upgrade
ci/prow/e2e-aws-ovn-kube-apiserver-rollout 00ca1c5 link false /test e2e-aws-ovn-kube-apiserver-rollout
ci/prow/e2e-azure-ovn-etcd-scaling 00ca1c5 link false /test e2e-azure-ovn-etcd-scaling
ci/prow/e2e-aws-ovn-single-node-serial 00ca1c5 link false /test e2e-aws-ovn-single-node-serial
ci/prow/e2e-metal-ipi-virtualmedia 00ca1c5 link false /test e2e-metal-ipi-virtualmedia
ci/prow/e2e-vsphere-ovn-dualstack-primaryv6 00ca1c5 link false /test e2e-vsphere-ovn-dualstack-primaryv6
ci/prow/e2e-gcp-disruptive 00ca1c5 link false /test e2e-gcp-disruptive
ci/prow/okd-e2e-gcp 00ca1c5 link false /test okd-e2e-gcp
ci/prow/e2e-aws-ovn-etcd-scaling 00ca1c5 link false /test e2e-aws-ovn-etcd-scaling
ci/prow/e2e-gcp-ovn-etcd-scaling 00ca1c5 link false /test e2e-gcp-ovn-etcd-scaling
ci/prow/e2e-aws-ovn-cgroupsv2 00ca1c5 link false /test e2e-aws-ovn-cgroupsv2
ci/prow/e2e-gcp-ovn-rt-upgrade 00ca1c5 link false /test e2e-gcp-ovn-rt-upgrade
ci/prow/e2e-metal-ipi-ovn-ipv6 00ca1c5 link true /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 6c48344 and 0 for PR HEAD 00ca1c5 in total

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@deads2k deads2k Awaiting requested review from deads2k

@sjenning sjenning Awaiting requested review from sjenning

Assignees

@sosiouxme sosiouxme

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@stbenjam @openshift-ci-robot @deads2k @sosiouxme