Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[OCPCLOUD-922] add alert for mhc in short circuit #886

Merged
merged 1 commit into from
Jul 26, 2021
Merged

[OCPCLOUD-922] add alert for mhc in short circuit #886

merged 1 commit into from
Jul 26, 2021

Conversation

elmiko
Copy link
Contributor

@elmiko elmiko commented Jul 9, 2021

This adds an alert for when an mhc has been in short-circuit for more
than 30 minutes, also adds some documentation about it.

ref: https://issues.redhat.com/browse/OCPCLOUD-922

michaelgugino
michaelgugino approved these changes Jul 9, 2021
View reviewed changes
Copy link
Contributor

@michaelgugino michaelgugino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 9, 2021
@elmiko
Copy link
Contributor Author

elmiko commented Jul 9, 2021

/cc @openshift/sre-alert-sme

ravitri
ravitri reviewed Jul 10, 2021
View reviewed changes
docs/user/Alerts.md Outdated Show resolved Hide resolved
@dofinn
Copy link

dofinn commented Jul 14, 2021

/lgtm

@elmiko
add alert for mhc in short circuit
6bc53e1 
This adds an alert for when an mhc has been in short-circuit for more
than 30 minutes, also adds some documentation about it.

ref: https://issues.redhat.com/browse/OCPCLOUD-922
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 14, 2021
@elmiko
Copy link
Contributor Author

elmiko commented Jul 14, 2021

updated

  • change message in alert to reflect disabled status of mhc
  • added more information to possible resolutions for this alert

cc @ravitri , let me know what you think of the new language in the alert doc

@elmiko
Copy link
Contributor Author

elmiko commented Jul 22, 2021

reminder: update openshift/enhancements#538 once this merges

@JoelSpeed
Copy link
Contributor

/approve
/lgtm
/retest-required

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 23, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 23, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JoelSpeed

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 23, 2021
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

8 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

20 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@JoelSpeed
Copy link
Contributor

/override ci/prow/e2e-aws-operator

Currently failing on a known perma failure which we are working on a fix for, this alert doesn't appear to be causing issues so I believe we are safe for this to merge

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 25, 2021

@JoelSpeed: Overrode contexts on behalf of JoelSpeed: ci/prow/e2e-aws-operator

In response to this:

/override ci/prow/e2e-aws-operator

Currently failing on a known perma failure which we are working on a fix for, this alert doesn't appear to be causing issues so I believe we are safe for this to merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 25, 2021 •
edited
Loading

@elmiko: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-azure-operator 6bc53e1 link /test e2e-azure-operator
ci/prow/e2e-gcp-operator 6bc53e1 link /test e2e-gcp-operator
ci/prow/e2e-vsphere 6bc53e1 link /test e2e-vsphere
ci/prow/e2e-metal-ipi-ovn-ipv6 6bc53e1 link /test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-aws-disruptive 6bc53e1 link /test e2e-aws-disruptive
ci/prow/e2e-vsphere-upgrade 6bc53e1 link /test e2e-vsphere-upgrade

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@JoelSpeed
Copy link
Contributor

/override ci/prow/e2e-aws-operator

Based on comment above

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 25, 2021

@JoelSpeed: Overrode contexts on behalf of JoelSpeed: ci/prow/e2e-aws-operator

In response to this:

/override ci/prow/e2e-aws-operator

Based on comment above

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

2 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 356e121 into openshift:master Jul 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ravitri ravitri ravitri left review comments

@dofinn dofinn dofinn left review comments

@michaelgugino michaelgugino michaelgugino approved these changes

@alexander-demicev alexander-demicev Awaiting requested review from alexander-demicev

Assignees

@michaelgugino michaelgugino

@JoelSpeed JoelSpeed

@dofinn dofinn

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@elmiko @dofinn @JoelSpeed @openshift-bot @ravitri @michaelgugino @openshift-merge-robot