Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support routing to external IPs behind service #2243

Merged
merged 1 commit into from
Feb 27, 2023

Conversation

jdn5126
Copy link
Contributor

@jdn5126 jdn5126 commented Jan 31, 2023

What type of PR is this?
bug

Which issue does this PR fix:
#1910

What does this PR do / Why do we need it:
This PR adds a new environment variable, AWS_EXTERNAL_SERVICE_CIDRS, where customers can specify IPV4 CIDRs outside of VPC that are reachable via service IP. This is necessary to route to these endpoints via service IP from secondary ENIs (#1910).

For each CIDR, an IP rule is installed (with a priority higher than the from-container rule) to route to the CIDR via main routing table. An example rule is:

1535:   from all to A.B.C.D lookup main

Currently, every time AWS_EXTERNAL_SERVICE_CIDRS is changed, the aws-node pod restarts, which is why the implementation for this function is in IPAMD node initialization. In the future, if a ConfigMap is used (#2102 is implemented), then operators will need to manually restart the aws-node container after modifying the environment variable.

If a solution can be implemented one day that dynamically detects this case and ensures proper routing, this environment variable will be deprecated.

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:
N/A

Testing done on this change:
Unit tests are added to cover new functions. I manually verified functionality and ensured that all existing integration tests pass. I considered adding an integration test case, but decided against it as it would require setting up communication outside of VPC or requiring node IPs to be public, both of which we want to avoid. The unit tests cover that we add and delete the IP rules as expected, and we know that routing works when IP rules are present.

Automation added to e2e:
N/A

Will this PR introduce any new dependencies?:
No

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
This PR will not break upgrades or downgrades. A running cluster has been tested.

Does this change require updates to the CNI daemonset config files to work?:
No

Does this PR introduce any user-facing change?:
Yes

Added AWS_EXTERNAL_SERVICE_CIDRS to resolve issues routing to endpoints outside of VPC backed by service.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jdn5126 jdn5126 requested a review from a team as a code owner January 31, 2023 17:32
@jdn5126 jdn5126 force-pushed the from_container_rule branch 2 times, most recently from 309c7da to 0cf7af2 Compare February 2, 2023 16:21
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@jdn5126 jdn5126 force-pushed the from_container_rule branch 2 times, most recently from 68bf466 to 9217208 Compare February 14, 2023 17:21
achevuru
achevuru previously approved these changes Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants