re-add support for sourcing endpoint URLs from service-specific env config #3568
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
ysaito1001
reviewed
Apr 9, 2024
aajtodd
reviewed
Apr 9, 2024
| /// | ||
| /// This is used internally to understand how to merge config structs while | ||
| /// respecting precedence of origins. | ||
| pub fn insert_origin(&mut self, setting: &'static str, origin: Origin) { |
There was a problem hiding this comment.
correctness: Do we want this to be mutable after it is built into SdkConfig?
There was a problem hiding this comment.
Good question. I think so, but I'm not sure. For now, I'll make it so that:
- SdkConfig's Builder exposes an
insert_originmethod. - SdkConfig itself only exposes a way to check origins.
We can add other stuff later if/when we need it.
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
ysaito1001
reviewed
Apr 9, 2024
Co-authored-by: ysaito1001 <awsaito@amazon.com>
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
ysaito1001
reviewed
Apr 9, 2024
aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/SdkConfigDecorator.kt
Show resolved
Hide resolved
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
Lastly, we probably need to have a changelog entry that revises or corrects release notes that we put out in https://github.com/awslabs/aws-sdk-rust/releases/tag/release-2024-04-08 and the discussion topic, since the support is currently for endpoint url. |
I updated the discussion. I think the Changelog entry is fine:
|
ysaito1001
approved these changes
Apr 10, 2024
1 task
github-merge-queue bot
pushed a commit
that referenced
this pull request
Oct 21, 2024
…3873) ## Motivation and Context <!--- Why is this change required? What problem does it solve? --> <!--- If it fixes an open issue, please link to the issue here --> awslabs/aws-sdk-rust#1193 ## Description This PR fixes a customer reported bug where the default chain doesn't respect `AWS_ENDPOINT_URL`/`AWS_ENDPOINT_URL_<SERVICE>` environment variables or the equivalents in AWS shared config (`~/.aws/config`). This fix is a little nuanced and frankly gross but there isn't a better option that I can see right now that isn't way more invasive. The crux of the issue is that when we implemented support for this feature ([1](#3568), [2](#3493), [3](#3488)) we really only made it work for clients created via [`ConfigLoader::load()`](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/lib.rs#L871). Internally the default chain credential provider constructs `STS` and `SSO` clients but it does so using [`ProviderConfig`](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/provider_config.rs#L36) by mapping this to `SdkConfig` via [`ProviderConfig::client_config()`](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/provider_config.rs#L199). This conversion is used in several places and it doesn't take any of the required logic into account to setup [`EnvServiceConfig`](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/lib.rs#L859-L862) which is what generated SDK's ultimately use to figure out the endpoint URL from either environment/profile ([example client](https://github.com/awslabs/aws-sdk-rust/blob/release-2024-10-09/sdk/sts/src/config.rs#L1214-L1221) which ultimately ends up in `EnvServiceConfig` [here](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/env_service_config.rs#L18)). The fix applied here is nuanced in that we update the conversion to provide a `EnvServiceConfig` but it relies on the profile to have been parsed already or else you'll get an empty/default profile. This generally works for the profile provider since the first thing we do is load the profile but in isolation it may not work as expected. I've added tests for STS to cover all cases but SSO credentials and token providers do NOT currently respect shared config endpoint URL keys. Fixing this is possible but involved since we require an `async` context to ensure a profile is loaded already and in many places where we construct `SdkConfig` from `ProviderConfig` we are in non async function. ## Testing Tested repro + additional integration tests ## Future This does _not_ fix awslabs/aws-sdk-rust#1194 which was discovered as a bug/gap. Fixing it would be outside the scope of this PR. SSO/token provider is instantiated sometimes before we have parsed a profile. This PR definitely fixes the STS provider for all configuration scenarios but the SSO related client usage may still have some edge cases when configured via profiles since we often instantiate them before parsing a profile. When we surveyed other SDKs there were several that failed to respect these variables and haven't received issues around this which leads me to believe this isn't likely a problem in practice (most likely due to SSO being used in local development most often where redirecting that endpoint doesn't make much sense anyway). ## Checklist - [X] For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "aws-sdk-rust" in the `applies_to` key. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
aws-sdk-rust-ci
pushed a commit
to awslabs/aws-sdk-rust
that referenced
this pull request
Oct 25, 2024
…overrides (#3873) ## Motivation and Context <!--- Why is this change required? What problem does it solve? --> <!--- If it fixes an open issue, please link to the issue here --> #1193 ## Description This PR fixes a customer reported bug where the default chain doesn't respect `AWS_ENDPOINT_URL`/`AWS_ENDPOINT_URL_<SERVICE>` environment variables or the equivalents in AWS shared config (`~/.aws/config`). This fix is a little nuanced and frankly gross but there isn't a better option that I can see right now that isn't way more invasive. The crux of the issue is that when we implemented support for this feature ([1](smithy-lang/smithy-rs#3568), [2](smithy-lang/smithy-rs#3493), [3](smithy-lang/smithy-rs#3488)) we really only made it work for clients created via [`ConfigLoader::load()`](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/lib.rs#L871). Internally the default chain credential provider constructs `STS` and `SSO` clients but it does so using [`ProviderConfig`](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/provider_config.rs#L36) by mapping this to `SdkConfig` via [`ProviderConfig::client_config()`](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/provider_config.rs#L199). This conversion is used in several places and it doesn't take any of the required logic into account to setup [`EnvServiceConfig`](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/lib.rs#L859-L862) which is what generated SDK's ultimately use to figure out the endpoint URL from either environment/profile ([example client](https://github.com/awslabs/aws-sdk-rust/blob/release-2024-10-09/sdk/sts/src/config.rs#L1214-L1221) which ultimately ends up in `EnvServiceConfig` [here](https://github.com/smithy-lang/smithy-rs/blob/release-2024-10-09/aws/rust-runtime/aws-config/src/env_service_config.rs#L18)). The fix applied here is nuanced in that we update the conversion to provide a `EnvServiceConfig` but it relies on the profile to have been parsed already or else you'll get an empty/default profile. This generally works for the profile provider since the first thing we do is load the profile but in isolation it may not work as expected. I've added tests for STS to cover all cases but SSO credentials and token providers do NOT currently respect shared config endpoint URL keys. Fixing this is possible but involved since we require an `async` context to ensure a profile is loaded already and in many places where we construct `SdkConfig` from `ProviderConfig` we are in non async function. ## Testing Tested repro + additional integration tests ## Future This does _not_ fix #1194 which was discovered as a bug/gap. Fixing it would be outside the scope of this PR. SSO/token provider is instantiated sometimes before we have parsed a profile. This PR definitely fixes the STS provider for all configuration scenarios but the SSO related client usage may still have some edge cases when configured via profiles since we often instantiate them before parsing a profile. When we surveyed other SDKs there were several that failed to respect these variables and haven't received issues around this which leads me to believe this isn't likely a problem in practice (most likely due to SSO being used in local development most often where redirecting that endpoint doesn't make much sense anyway). ## Checklist - [X] For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "aws-sdk-rust" in the `applies_to` key. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The service env config work I did was flawed in that it didn't respect the precedence of programmatic config. This PR
adds a way of tracking the precedence up to the point of converting the SdkConfig into a service config. Now, env config will only be resolved when config was not set programmatically. I added tests to
aws-configfor the origin tracking to ensure it works.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.