Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix EC2MetadataInstanceInfo ENI calculation #2065

Merged
merged 1 commit into from
Jun 20, 2024

Conversation

AndrewSirenko
Copy link
Contributor

@AndrewSirenko AndrewSirenko commented Jun 17, 2024

Is this a bug fix or adding new feature?
bug fix

What is this PR about? / Why do we need it?
Fixes #2064; Fixes a regression in calculating currently attached ENIs in our EC2MetadataInstanceInfo function to calculate CSINode allocatable count when not using reservedVolumeAttachments.

Instead of checking for newlines, we should check for counts of a relevant regex, to avoid inconsistencies when IMDS returns or doesn't return a newline at end of response.

What testing is done?
Can confirm that there is no newline when querying IMDS for number of attached ENIs:

> && curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/network/interfaces/macs
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    56  100    56    0     0  46128      0 --:--:-- --:--:-- --:--:-- 56000
0e:1c:7d:81:2b:19/
0e:8c:22:a2:16:ef/sh-4.2$

MAC address regex is sufficient across Linux and Windows CSI Driver deploys.

Open Question

Should we be smarter here and regex for a mac address instead of newlines? Seems like other IMDS endpoints DO end with a /n... Yes, new commit incoming.

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 17, 2024
@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Jun 17, 2024
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 17, 2024
Copy link

github-actions bot commented Jun 17, 2024

Code Coverage Diff

File Old Coverage New Coverage Delta
github.com/kubernetes-sigs/aws-ebs-csi-driver/pkg/cloud/metadata/ec2.go 94.3% 94.4% 0.1
github.com/kubernetes-sigs/aws-ebs-csi-driver/pkg/util/util.go 62.3% 63.6% 1.4

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 17, 2024
@AndrewSirenko AndrewSirenko changed the title [WIP] Fix EC2MetadataInstanceInfo ENI calculation Fix EC2MetadataInstanceInfo ENI calculation Jun 17, 2024
@AndrewSirenko AndrewSirenko marked this pull request as ready for review June 17, 2024 17:52
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 17, 2024
@AndrewSirenko
Copy link
Contributor Author

/retest

1 similar comment
@ConnorJC3
Copy link
Contributor

/retest

pkg/util/util.go Outdated Show resolved Hide resolved
@AndrewSirenko
Copy link
Contributor Author

AndrewSirenko commented Jun 17, 2024

Looking like windows failure isn't flake... Will go debug and add unit test for windows case (and perhaps snow device and metal instance cases?)

Worst case we have old version of this PR for release to at least fix regression.

@AndrewSirenko
Copy link
Contributor Author

/retest

@AndrewSirenko
Copy link
Contributor Author

#2063 also failing upon a retest of windows job, so looks like not related to current PR. Will investigate during working hours. A quick google of errors shows a possible certs issue on test cluster? But may be red herring. EKS remote error: tls: internal error when running kubectl logs command | AWS re:Post

Manually testing IMDS query on windows 2019 and 2022 shows that regex would work correctly.

PS C:\C\2b63e1adda895b3754e1fac883a865627c50de7743be651596b1cfb9e02283d7> Invoke-RestMethod -Headers @{"X-aws-ec2-metadata-token" = $token} -Method GET -Uri http://169.254.169.254/latest/meta-data/network/interfaces/macs
02:17:f1:05:9d:33/
PS C:\C\2b63e1adda895b3754e1fac883a865627c50de7743be651596b1cfb9e02283d7>

@AndrewSirenko
Copy link
Contributor Author

Looks like #2066 is passing

@ConnorJC3
Copy link
Contributor

/retest

1 similar comment
@torredil
Copy link
Member

/retest

Copy link
Member

@torredil torredil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround on the fix, the regex approach is much more resilient than the current new line tech.

Manually tested changes:

  • Windows node with 2 ENIs:
kubectl logs ebs-csi-node-windows-z2xw2 -n kube-system                                                                                               
Defaulted container "ebs-plugin" out of: ebs-plugin, node-driver-registrar, liveness-probe
I0618 18:36:19.041526    3192 main.go:157] "Initializing metadata"
I0618 18:36:19.288941    3192 ec2.go:92] "Number of attached ENIs" attachedENIs=2 Enis=<
        0a:03:b0:13:df:8b/
        0a:30:70:82:cf:6f/
 >
  • Linux node with 1 ENI:
kubectl logs ebs-csi-node-bwfv5 -n kube-system                                                                                                       
Defaulted container "ebs-plugin" out of: ebs-plugin, node-driver-registrar, liveness-probe
I0618 18:36:14.821441       1 main.go:157] "Initializing metadata"
I0618 18:36:14.824222       1 ec2.go:92] "Number of attached ENIs" attachedENIs=1 Enis="0a:fa:d3:89:fa:b3/"

pkg/cloud/metadata/ec2.go Show resolved Hide resolved
@ConnorJC3
Copy link
Contributor

/retest

@AndrewSirenko
Copy link
Contributor Author

/test pull-aws-ebs-csi-driver-test-e2e-external-eks-windows

Copy link
Contributor

@ConnorJC3 ConnorJC3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 20, 2024
@torredil
Copy link
Member

/retest

Copy link
Member

@torredil torredil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: torredil

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 20, 2024
@k8s-ci-robot k8s-ci-robot merged commit 2ace110 into kubernetes-sigs:master Jun 20, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CSI driver miscounting ENIs on instance?
4 participants