Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AWS Health event support for QP mode #510

Merged
merged 10 commits into from
Oct 27, 2021
Merged

Conversation

AustinSiu
Copy link
Contributor

@AustinSiu AustinSiu commented Oct 14, 2021

Please do not merge, WIP

Description of changes: Introduce new event handler for AWS Health Scheduled Change events. AWS Health events can specify more than 1 host, which differs from currently-supported events. To support this, I'm introducing a wrapper around interruption events and related errors to more-easily manage the list.

Related issue: #293

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link
Contributor

@brycahta brycahta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 1st PR 🎉 🎉

pkg/monitor/sqsevent/health-maintenance-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/health-maintenance-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
test/e2e/aws-health-maintenance-event-sqs-test Outdated Show resolved Hide resolved
go.mod Show resolved Hide resolved
Copy link
Contributor

@bwagner5 bwagner5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on this!! Just a few minor comments

pkg/monitor/sqsevent/health-maintenance-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/health-maintenance-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/health-maintenance-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/health-maintenance-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
@jillmon jillmon linked an issue Oct 19, 2021 that may be closed by this pull request
pkg/monitor/sqsevent/scheduled-change-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
test/e2e/scheduled-change-event-sqs-test Outdated Show resolved Hide resolved
@AustinSiu AustinSiu changed the title [WIP] Add AWS Health event support for QP mode Add AWS Health event support for QP mode Oct 26, 2021
@AustinSiu AustinSiu marked this pull request as ready for review October 26, 2021 15:29
pkg/monitor/sqsevent/scheduled-change-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/scheduled-change-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
}

// processInterruptionEvents takes interruption event wrappers and sends interruption events to the passed-in channel
func (m SQSMonitor) processInterruptionEvents(interruptionEventWrappers []InterruptionEventWrapper, message *sqs.Message) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice improvement in the error handling-- much cleaner 👍

brycahta
brycahta previously approved these changes Oct 26, 2021
Copy link
Contributor

@brycahta brycahta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

Copy link
Contributor

@bwagner5 bwagner5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few little things to clean up. Nice work on this feature!

pkg/monitor/sqsevent/scheduled-change-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/scheduled-change-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/scheduled-change-event.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
pkg/monitor/sqsevent/sqs-monitor.go Outdated Show resolved Hide resolved
Copy link
Contributor

@brycahta brycahta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link
Contributor

@bwagner5 bwagner5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm nice work!


$ aws events put-rule \
--name MyK8sScheduledChangeRule \
--event-pattern "{\"source\": [\"aws.health\"],\"detail-type\": [\"AWS Health Event\"]}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason this event pattern isn't more specific? Should this be limited to EC2 scheduled changes since the code ignores all other events?

{
  "source": [
    "aws.health"
  ],
  "detail-type": [
    "AWS Health Event"
  ],
  "detail": {
    "service": [
      "EC2"
    ],
    "eventTypeCategory": [
      "scheduledChange"
    ]
  }
}

Apologies for commenting on a closed PR. I'm trying to upgrade NTH and need to understand the new event rules that should be added.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also add "region": ["us-east-1"] or whatever region you're using for the cluster. Since AWS Health events are global and at least want the filter to EC2 scheduled changes for the region you're using.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gabegorelick can you open an issue with your thoughts on how we should constrain the rule? Also, if you'd like to PR an update to the README and run some tests, that would be great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Queue-Processor should catch AWS Health Events
6 participants