Only bail to AttachVolume if volume is detached #2183
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Connor Catlett <conncatl@amazon.com>
k8s-ci-robot
added
cncf-cla: yes
Code Coverage DiffThis PR does not change the code coverage |
k8s-ci-robot
added
the
lgtm
AndrewSirenko
approved these changes
Oct 14, 2024
ElijahQuinones
approved these changes
Oct 14, 2024
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: AndrewSirenko The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Is this a bug fix or adding new feature?
Bug fix
What is this PR about? / Why do we need it?
In
WaitForAttachmentState, we bail to callingAttachVolumeand immediately returning an error if the volume was reported as attached byDescribeInstances(i.e. ifalreadyAssignedis true), but the volume returns a state other thanattached. This check exists to guard against the case whereDescribeInstancesis incorrect, bailing early allows us to get the attachment moving and avoid sitting in an infinite loop until a timeout occurs.However, this is incorrect - we should only perform this action if the volume is
detached, not all states other thanattached. This is for two reasons:AttachVolumecall when the volume is in a state other thandetachedwill almost always fail.DescribeInstancesbut still report asattachingviaDescribeVolumesfor a short period of time. In this case, we repeatedly error out at the earliest possible opportunity, preventingWaitForAttachmentState's exponential backoff from working and peppering the AWS API with spuriousAttachVolumecalls.What testing is done?
Added new unit test. Note that although depending on the test name is a horrible practice we should remove from this test, I have not spent the significant effort that would be required in this small PR to fix it.