-
Notifications
You must be signed in to change notification settings - Fork 187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add: request information header interceptors and plugins #2641
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
40dd736
add: request information header interceptors and plugins
Velfi 86f7807
update: in response to PR comments
Velfi a88ccad
Merge branch 'main' into add/retry-information-headers
Velfi 9388215
fix: clippy lints
Velfi d9c5da3
add: http types to approved external types for aws-runtime crate
Velfi 3c10c5a
Merge remote-tracking branch 'origin/main' into add/retry-information…
Velfi 3ae2d90
add: broken tests
Velfi 8311574
add test ensuring support for all valid request pair numbers, charact…
Velfi 0b8b760
Merge remote-tracking branch 'origin/main' into add/retry-information…
Velfi 3ed5760
fix test and codegen broken by merge from main
Velfi b478ddd
fix issue with tracing minimal version by upgrading tracing dep
Velfi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,224 @@ | ||
/* | ||
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
* SPDX-License-Identifier: Apache-2.0 | ||
*/ | ||
|
||
use aws_smithy_runtime::client::orchestrator::interceptors::{RequestAttempts, ServiceClockSkew}; | ||
use aws_smithy_runtime_api::client::interceptors::context::phase::BeforeTransmit; | ||
use aws_smithy_runtime_api::client::interceptors::{BoxError, Interceptor, InterceptorContext}; | ||
use aws_smithy_runtime_api::config_bag::ConfigBag; | ||
use aws_smithy_types::date_time::Format; | ||
use aws_smithy_types::retry::RetryConfig; | ||
use aws_smithy_types::timeout::TimeoutConfig; | ||
use aws_smithy_types::DateTime; | ||
use http::{HeaderName, HeaderValue}; | ||
use std::borrow::Cow; | ||
use std::time::{Duration, SystemTime}; | ||
|
||
#[allow(clippy::declare_interior_mutable_const)] // we will never mutate this | ||
const AMZ_SDK_REQUEST: HeaderName = HeaderName::from_static("amz-sdk-request"); | ||
|
||
/// Generates and attaches a request header that communicates request-related metadata. | ||
/// Examples include: | ||
/// | ||
/// - When the client will time out this request. | ||
/// - How many times the request has been retried. | ||
/// - The maximum number of retries that the client will attempt. | ||
#[non_exhaustive] | ||
#[derive(Debug, Default)] | ||
pub struct RequestInfoInterceptor {} | ||
|
||
impl RequestInfoInterceptor { | ||
/// Creates a new `RequestInfoInterceptor` | ||
pub fn new() -> Self { | ||
RequestInfoInterceptor {} | ||
} | ||
} | ||
|
||
impl RequestInfoInterceptor { | ||
fn build_attempts_pair( | ||
&self, | ||
cfg: &ConfigBag, | ||
) -> Option<(Cow<'static, str>, Cow<'static, str>)> { | ||
let request_attempts = cfg | ||
.get::<RequestAttempts>() | ||
.map(|r_a| r_a.attempts()) | ||
.unwrap_or(1); | ||
let request_attempts = request_attempts.to_string(); | ||
Some((Cow::Borrowed("attempt"), Cow::Owned(request_attempts))) | ||
} | ||
|
||
fn build_max_attempts_pair( | ||
&self, | ||
cfg: &ConfigBag, | ||
) -> Option<(Cow<'static, str>, Cow<'static, str>)> { | ||
// TODO(enableNewSmithyRuntime) What config will we actually store in the bag? Will it be a whole config or just the max_attempts part? | ||
if let Some(retry_config) = cfg.get::<RetryConfig>() { | ||
let max_attempts = retry_config.max_attempts().to_string(); | ||
Some((Cow::Borrowed("max"), Cow::Owned(max_attempts))) | ||
} else { | ||
None | ||
} | ||
} | ||
|
||
fn build_ttl_pair(&self, cfg: &ConfigBag) -> Option<(Cow<'static, str>, Cow<'static, str>)> { | ||
let timeout_config = cfg.get::<TimeoutConfig>()?; | ||
let socket_read = timeout_config.read_timeout()?; | ||
let estimated_skew: Duration = cfg.get::<ServiceClockSkew>().cloned()?.into(); | ||
let current_time = SystemTime::now(); | ||
let ttl = current_time.checked_add(socket_read + estimated_skew)?; | ||
let timestamp = DateTime::from(ttl); | ||
let formatted_timestamp = timestamp | ||
.fmt(Format::DateTime) | ||
.expect("the resulting DateTime will always be valid"); | ||
|
||
Some((Cow::Borrowed("ttl"), Cow::Owned(formatted_timestamp))) | ||
} | ||
} | ||
|
||
impl Interceptor for RequestInfoInterceptor { | ||
fn modify_before_transmit( | ||
&self, | ||
context: &mut InterceptorContext<BeforeTransmit>, | ||
cfg: &mut ConfigBag, | ||
) -> Result<(), BoxError> { | ||
let mut pairs = RequestPairs::new(); | ||
if let Some(pair) = self.build_attempts_pair(cfg) { | ||
pairs = pairs.with_pair(pair); | ||
} | ||
if let Some(pair) = self.build_max_attempts_pair(cfg) { | ||
pairs = pairs.with_pair(pair); | ||
} | ||
if let Some(pair) = self.build_ttl_pair(cfg) { | ||
pairs = pairs.with_pair(pair); | ||
} | ||
|
||
let headers = context.request_mut().headers_mut(); | ||
headers.insert(AMZ_SDK_REQUEST, pairs.try_into_header_value()?); | ||
|
||
Ok(()) | ||
} | ||
} | ||
|
||
/// A builder for creating a `RequestPairs` header value. `RequestPairs` is used to generate a | ||
/// retry information header that is sent with every request. The information conveyed by this | ||
/// header allows services to anticipate whether a client will time out or retry a request. | ||
#[derive(Default, Debug)] | ||
pub struct RequestPairs { | ||
inner: Vec<(Cow<'static, str>, Cow<'static, str>)>, | ||
} | ||
|
||
impl RequestPairs { | ||
/// Creates a new `RequestPairs` builder. | ||
pub fn new() -> Self { | ||
Default::default() | ||
} | ||
|
||
/// Adds a pair to the `RequestPairs` builder. | ||
/// Only strings that can be converted to header values are considered valid. | ||
pub fn with_pair( | ||
mut self, | ||
pair: (impl Into<Cow<'static, str>>, impl Into<Cow<'static, str>>), | ||
) -> Self { | ||
let pair = (pair.0.into(), pair.1.into()); | ||
self.inner.push(pair); | ||
self | ||
} | ||
|
||
/// Converts the `RequestPairs` builder into a `HeaderValue`. | ||
pub fn try_into_header_value(self) -> Result<HeaderValue, BoxError> { | ||
self.try_into() | ||
} | ||
} | ||
|
||
impl TryFrom<RequestPairs> for HeaderValue { | ||
type Error = BoxError; | ||
|
||
fn try_from(value: RequestPairs) -> Result<Self, BoxError> { | ||
let mut pairs = String::new(); | ||
for (key, value) in value.inner { | ||
if !pairs.is_empty() { | ||
pairs.push_str("; "); | ||
} | ||
|
||
pairs.push_str(&key); | ||
pairs.push('='); | ||
pairs.push_str(&value); | ||
continue; | ||
} | ||
HeaderValue::from_str(&pairs).map_err(Into::into) | ||
} | ||
} | ||
|
||
#[cfg(test)] | ||
mod tests { | ||
use super::RequestInfoInterceptor; | ||
use crate::request_info::RequestPairs; | ||
use aws_smithy_http::body::SdkBody; | ||
use aws_smithy_runtime::client::orchestrator::interceptors::RequestAttempts; | ||
use aws_smithy_runtime_api::client::interceptors::context::phase::BeforeTransmit; | ||
use aws_smithy_runtime_api::client::interceptors::{Interceptor, InterceptorContext}; | ||
use aws_smithy_runtime_api::config_bag::ConfigBag; | ||
use aws_smithy_runtime_api::type_erasure::TypedBox; | ||
use aws_smithy_types::retry::RetryConfig; | ||
use aws_smithy_types::timeout::TimeoutConfig; | ||
use http::HeaderValue; | ||
use std::time::Duration; | ||
|
||
fn expect_header<'a>( | ||
context: &'a InterceptorContext<BeforeTransmit>, | ||
header_name: &str, | ||
) -> &'a str { | ||
context | ||
.request() | ||
.headers() | ||
.get(header_name) | ||
.unwrap() | ||
.to_str() | ||
.unwrap() | ||
} | ||
|
||
#[test] | ||
fn test_request_pairs_for_initial_attempt() { | ||
let context = InterceptorContext::<()>::new(TypedBox::new("doesntmatter").erase()); | ||
let mut context = context.into_serialization_phase(); | ||
context.set_request(http::Request::builder().body(SdkBody::empty()).unwrap()); | ||
|
||
let mut config = ConfigBag::base(); | ||
config.put(RetryConfig::standard()); | ||
config.put( | ||
TimeoutConfig::builder() | ||
.read_timeout(Duration::from_secs(30)) | ||
.build(), | ||
); | ||
config.put(RequestAttempts::new()); | ||
|
||
let _ = context.take_input(); | ||
let mut context = context.into_before_transmit_phase(); | ||
let interceptor = RequestInfoInterceptor::new(); | ||
interceptor | ||
.modify_before_transmit(&mut context, &mut config) | ||
.unwrap(); | ||
|
||
assert_eq!( | ||
expect_header(&context, "amz-sdk-request"), | ||
"attempt=0; max=3" | ||
); | ||
} | ||
|
||
#[test] | ||
fn test_header_value_from_request_pairs_supports_all_valid_characters() { | ||
// The list of valid characters is defined by an internal-only spec. | ||
let rp = RequestPairs::new() | ||
.with_pair(("allowed-symbols", "!#$&'*+-.^_`|~")) | ||
.with_pair(("allowed-digits", "01234567890")) | ||
.with_pair(( | ||
"allowed-characters", | ||
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ", | ||
)) | ||
.with_pair(("allowed-whitespace", " \t")); | ||
let _header_value: HeaderValue = rp | ||
.try_into() | ||
.expect("request pairs can be converted into valid header value."); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
55 changes: 55 additions & 0 deletions
55
...codegen/src/main/kotlin/software/amazon/smithy/rustsdk/RetryInformationHeaderDecorator.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
/* | ||
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
* SPDX-License-Identifier: Apache-2.0 | ||
*/ | ||
|
||
package software.amazon.smithy.rustsdk | ||
|
||
import software.amazon.smithy.rust.codegen.client.smithy.ClientCodegenContext | ||
import software.amazon.smithy.rust.codegen.client.smithy.customize.ClientCodegenDecorator | ||
import software.amazon.smithy.rust.codegen.client.smithy.generators.ServiceRuntimePluginCustomization | ||
import software.amazon.smithy.rust.codegen.client.smithy.generators.ServiceRuntimePluginSection | ||
import software.amazon.smithy.rust.codegen.core.rustlang.Writable | ||
import software.amazon.smithy.rust.codegen.core.rustlang.rust | ||
import software.amazon.smithy.rust.codegen.core.rustlang.writable | ||
import software.amazon.smithy.rust.codegen.core.smithy.RuntimeType | ||
import software.amazon.smithy.rust.codegen.core.util.letIf | ||
|
||
class RetryInformationHeaderDecorator : ClientCodegenDecorator { | ||
override val name: String = "RetryInformationHeader" | ||
override val order: Byte = 10 | ||
|
||
override fun serviceRuntimePluginCustomizations( | ||
codegenContext: ClientCodegenContext, | ||
baseCustomizations: List<ServiceRuntimePluginCustomization>, | ||
): List<ServiceRuntimePluginCustomization> = | ||
baseCustomizations.letIf(codegenContext.smithyRuntimeMode.generateOrchestrator) { | ||
it + listOf(AddRetryInformationHeaderInterceptors(codegenContext)) | ||
} | ||
} | ||
|
||
private class AddRetryInformationHeaderInterceptors(codegenContext: ClientCodegenContext) : | ||
ServiceRuntimePluginCustomization() { | ||
private val runtimeConfig = codegenContext.runtimeConfig | ||
private val smithyRuntime = RuntimeType.smithyRuntime(runtimeConfig) | ||
private val awsRuntime = AwsRuntimeType.awsRuntime(runtimeConfig) | ||
|
||
override fun section(section: ServiceRuntimePluginSection): Writable = writable { | ||
if (section is ServiceRuntimePluginSection.AdditionalConfig) { | ||
// Track the latency between client and server. | ||
section.registerInterceptor(runtimeConfig, this) { | ||
rust("#T::new()", smithyRuntime.resolve("client::orchestrator::interceptors::ServiceClockSkewInterceptor")) | ||
} | ||
|
||
// Track the number of request attempts made. | ||
section.registerInterceptor(runtimeConfig, this) { | ||
rust("#T::new()", smithyRuntime.resolve("client::orchestrator::interceptors::RequestAttemptsInterceptor")) | ||
} | ||
|
||
// Add request metadata to outgoing requests. Sets a header. | ||
section.registerInterceptor(runtimeConfig, this) { | ||
rust("#T::new()", awsRuntime.resolve("request_info::RequestInfoInterceptor")) | ||
} | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this right? Seems like it needs to factor in multiple possible timeouts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The internal-only spec for this specifies the calculation this way.