Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1986003: Switch kubeconfig retries to use backoff #26407

Merged
merged 1 commit into from
Aug 19, 2021

Conversation

soltysh
Copy link
Contributor

@soltysh soltysh commented Aug 18, 2021

This is a followup to #26377 but I'm switching this time to use backoff, so it'll retry more but it'll do that after 2, 10 and 50 seconds.

In https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/26401/pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6/1427903228267204608 I've noticed we did retry, but every 2s which was too fast 😞

/assign @aojea

This logic will retry the kubeconfig check after 2, 10 and finally after 50s.
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 18, 2021

@soltysh: This pull request references Bugzilla bug 1986003, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.9.0) matches configured target release for branch (4.9.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @wangke19

In response to this:

Bug 1986003: Switch kubeconfig retries to use backoff

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Aug 18, 2021
@openshift-ci openshift-ci bot requested a review from wangke19 August 18, 2021 10:34
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 18, 2021

@soltysh: This pull request references Bugzilla bug 1986003, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.9.0) matches configured target release for branch (4.9.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @wangke19

In response to this:

Bug 1986003: Switch kubeconfig retries to use backoff

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot requested review from bparees and gabemontero August 18, 2021 10:36
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 18, 2021
@aojea
Copy link
Contributor

aojea commented Aug 18, 2021

... and detecting the pods is running and ready before doing the check?
there are already helpers for doing that

@stbenjam
Copy link
Member

https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.9-e2e-metal-ipi-ovn-ipv6/1428006633924988928 failed the local kubeconfig test today, which was after #26377 merged -- should this PR fix those?

@stbenjam
Copy link
Member

/retest-required

matched, _ := regexp.MatchString("The connection to the server .+ was refused - did you specify the right host or port", out)
return !matched, err
if err != nil {
return fmt.Errorf(out)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is The connection to the server .+ was refused - did you specify the right host or port reported in out?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I see it was

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I had to switch from using err to out due to how wait.Retry uses 2 separate functions for action and decision about retry.

@aojea
Copy link
Contributor

aojea commented Aug 18, 2021

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 18, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 18, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aojea, soltysh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

2 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@stbenjam
Copy link
Member

/retest-required

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

3 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@soltysh
Copy link
Contributor Author

soltysh commented Aug 19, 2021

/override ci/prow/e2e-agnostic-cmd
since that one is optional and is being actively worked on
/override ci/prow/e2e-aws-single-node
since it's also optional

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 19, 2021

@soltysh: Overrode contexts on behalf of soltysh: ci/prow/e2e-agnostic-cmd, ci/prow/e2e-aws-single-node

In response to this:

/override ci/prow/e2e-agnostic-cmd
since that one is optional and is being actively worked on
/override ci/prow/e2e-aws-single-node
since it's also optional

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@soltysh
Copy link
Contributor Author

soltysh commented Aug 19, 2021

/test e2e-gcp-upgrade

@openshift-merge-robot openshift-merge-robot merged commit 5306ae6 into openshift:master Aug 19, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 19, 2021

@soltysh: Bugzilla bug 1986003 is in an unrecognized state (MODIFIED) and will not be moved to the MODIFIED state.

In response to this:

Bug 1986003: Switch kubeconfig retries to use backoff

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants