`GetObjectFluentBuilder.if_modified_since` has no effect #818

cowlicks · 2023-06-02T15:34:37Z

Describe the bug

When using GetObjectFluentBuilder.if_modified_since or set_if_modified_since to get an object, the object is always returned. I tried both methods, and tried setting the input to be both before and after the last_modified date of the object (from GetObjectOutput.last_modified). In every case, the object is returned.

Expected Behavior

The set_if_modified_since and if_modified_methods should only return an object if the object was modified after the provided input.

Current Behavior

the set_if_modified_since and if_modified_methods do nothing.

Reproduction Steps

#[tokio::test]
async fn bug_report() -> Result<()> {
    dotenvy::dotenv().ok();
    let config = ::aws_config::load_from_env().await;
    let client = aws_sdk_s3::Client::new(&config);
    let key = "my_key_that_exists.json";
    let bucket = "my-bucket";

    let if_modified_since_input: aws_sdk_s3::primitives::DateTime =
        std::time::SystemTime::now().into();
    let resp = client
        .get_object()
        .bucket(bucket)
        .key(key)
        .if_modified_since(if_modified_since_input)
        .send()
        .await?;
    assert!(resp.last_modified.unwrap() > if_modified_since_input);
    dbg!(resp);
    Ok(())
}

Possible Solution

No response

Additional Information/Context

No response

Version

├── aws-config v0.55.3
│   ├── aws-credential-types v0.55.3
│   │   ├── aws-smithy-async v0.55.3
│   │   ├── aws-smithy-types v0.55.3
│   ├── aws-http v0.55.3
│   │   ├── aws-credential-types v0.55.3 (*)
│   │   ├── aws-smithy-http v0.55.3
│   │   │   ├── aws-smithy-eventstream v0.55.3
│   │   │   │   ├── aws-smithy-types v0.55.3 (*)
│   │   │   ├── aws-smithy-types v0.55.3 (*)
│   │   ├── aws-smithy-types v0.55.3 (*)
│   │   ├── aws-types v0.55.3
│   │   │   ├── aws-credential-types v0.55.3 (*)
│   │   │   ├── aws-smithy-async v0.55.3 (*)
│   │   │   ├── aws-smithy-client v0.55.3
│   │   │   │   ├── aws-smithy-async v0.55.3 (*)
│   │   │   │   ├── aws-smithy-http v0.55.3 (*)
│   │   │   │   ├── aws-smithy-http-tower v0.55.3
│   │   │   │   │   ├── aws-smithy-http v0.55.3 (*)
│   │   │   │   │   ├── aws-smithy-types v0.55.3 (*)
│   │   │   │   ├── aws-smithy-types v0.55.3 (*)
│   │   │   ├── aws-smithy-http v0.55.3 (*)
│   │   │   ├── aws-smithy-types v0.55.3 (*)
│   ├── aws-sdk-sso v0.28.0
│   │   ├── aws-credential-types v0.55.3 (*)
│   │   ├── aws-endpoint v0.55.3
│   │   │   ├── aws-smithy-http v0.55.3 (*)
│   │   │   ├── aws-smithy-types v0.55.3 (*)
│   │   │   ├── aws-types v0.55.3 (*)
│   │   ├── aws-http v0.55.3 (*)
│   │   ├── aws-sig-auth v0.55.3
│   │   │   ├── aws-credential-types v0.55.3 (*)
│   │   │   ├── aws-sigv4 v0.55.3
│   │   │   │   ├── aws-smithy-eventstream v0.55.3 (*)
│   │   │   │   ├── aws-smithy-http v0.55.3 (*)
│   │   │   ├── aws-smithy-eventstream v0.55.3 (*)
│   │   │   ├── aws-smithy-http v0.55.3 (*)
│   │   │   ├── aws-types v0.55.3 (*)
│   │   ├── aws-smithy-async v0.55.3 (*)
│   │   ├── aws-smithy-client v0.55.3 (*)
│   │   ├── aws-smithy-http v0.55.3 (*)
│   │   ├── aws-smithy-http-tower v0.55.3 (*)
│   │   ├── aws-smithy-json v0.55.3
│   │   │   └── aws-smithy-types v0.55.3 (*)
│   │   ├── aws-smithy-types v0.55.3 (*)
│   │   ├── aws-types v0.55.3 (*)
│   ├── aws-sdk-sts v0.28.0
│   │   ├── aws-credential-types v0.55.3 (*)
│   │   ├── aws-endpoint v0.55.3 (*)
│   │   ├── aws-http v0.55.3 (*)
│   │   ├── aws-sig-auth v0.55.3 (*)
│   │   ├── aws-smithy-async v0.55.3 (*)
│   │   ├── aws-smithy-client v0.55.3 (*)
│   │   ├── aws-smithy-http v0.55.3 (*)
│   │   ├── aws-smithy-http-tower v0.55.3 (*)
│   │   ├── aws-smithy-json v0.55.3 (*)
│   │   ├── aws-smithy-query v0.55.3
│   │   │   ├── aws-smithy-types v0.55.3 (*)
│   │   ├── aws-smithy-types v0.55.3 (*)
│   │   ├── aws-smithy-xml v0.55.3
│   │   ├── aws-types v0.55.3 (*)
│   ├── aws-smithy-async v0.55.3 (*)
│   ├── aws-smithy-client v0.55.3 (*)
│   ├── aws-smithy-http v0.55.3 (*)
│   ├── aws-smithy-http-tower v0.55.3 (*)
│   ├── aws-smithy-json v0.55.3 (*)
│   ├── aws-smithy-types v0.55.3 (*)
│   ├── aws-types v0.55.3 (*)
├── aws-sdk-s3 v0.28.0
│   ├── aws-credential-types v0.55.3 (*)
│   ├── aws-endpoint v0.55.3 (*)
│   ├── aws-http v0.55.3 (*)
│   ├── aws-sig-auth v0.55.3 (*)
│   ├── aws-sigv4 v0.55.3 (*)
│   ├── aws-smithy-async v0.55.3 (*)
│   ├── aws-smithy-checksums v0.55.3
│   │   ├── aws-smithy-http v0.55.3 (*)
│   │   ├── aws-smithy-types v0.55.3 (*)
│   ├── aws-smithy-client v0.55.3 (*)
│   ├── aws-smithy-eventstream v0.55.3 (*)
│   ├── aws-smithy-http v0.55.3 (*)
│   ├── aws-smithy-http-tower v0.55.3 (*)
│   ├── aws-smithy-json v0.55.3 (*)
│   ├── aws-smithy-types v0.55.3 (*)
│   ├── aws-smithy-xml v0.55.3 (*)
│   ├── aws-types v0.55.3 (*)

Environment details (OS name and version, etc.)

arch linux

Logs

running 1 test
[2023-06-02T15:30:20Z INFO  aws_config::meta::region] load_region; provider=EnvironmentVariableRegionProvider { env: Env(Real) }
[2023-06-02T15:30:20Z DEBUG aws_config::fs_util] loaded home directory src="HOME"
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] load_config_file; file=Default(Config)
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] performing home directory substitution home="/home/<REDACTED>" path="~/.aws/config"
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] home directory expanded before="~/.aws/config" after="/home/<REDACTED>/.aws/config"
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] config file loaded path=Some("/home/<REDACTED>/.aws/config") size=47
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] load_config_file; file=Default(Credentials)
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] performing home directory substitution home="/home/<REDACTED>" path="~/.aws/credentials"
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] home directory expanded before="~/.aws/credentials" after="/home/<REDACTED>/.aws/credentials"
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] config file not found path=~/.aws/credentials
[2023-06-02T15:30:20Z DEBUG aws_config::profile::parser::source] config file loaded path=Some("/home/<REDACTED>/.aws/credentials") size=0
[2023-06-02T15:30:20Z DEBUG tracing::span] build_profile_provider;
[2023-06-02T15:30:20Z DEBUG hyper_rustls::config] with_native_roots processed 143 valid and 0 invalid certs
[2023-06-02T15:30:20Z DEBUG aws_smithy_client] send_operation;
[2023-06-02T15:30:20Z DEBUG aws_smithy_client] send_operation; operation="GetObject"
[2023-06-02T15:30:20Z DEBUG aws_smithy_client] send_operation; service="s3"
[2023-06-02T15:30:20Z DEBUG aws_smithy_http_tower::map_request] map_request; name="resolve_endpoint"
[2023-06-02T15:30:20Z DEBUG aws_smithy_http_tower::map_request] map_request; name="resolve_endpoint"
[2023-06-02T15:30:20Z DEBUG aws_smithy_http_tower::map_request] map_request; name="generate_user_agent"
[2023-06-02T15:30:20Z DEBUG aws_smithy_http_tower::map_request] async_map_request; name="retrieve_credentials"
[2023-06-02T15:30:20Z INFO  tracing::span] lazy_load_credentials;
[2023-06-02T15:30:20Z DEBUG aws_config::default_provider::credentials] provide_credentials; provider=default_chain
[2023-06-02T15:30:20Z DEBUG aws_config::meta::credentials::chain] load_credentials; provider=Environment
[2023-06-02T15:30:20Z DEBUG aws_config::meta::credentials::chain] loaded credentials provider=Environment
[2023-06-02T15:30:20Z INFO  aws_credential_types::cache::lazy_caching] credentials cache miss occurred; added new AWS credentials (took 70.706µs)
[2023-06-02T15:30:20Z DEBUG aws_credential_types::cache::lazy_caching] loaded credentials
[2023-06-02T15:30:20Z DEBUG aws_smithy_http_tower::map_request] map_request; name="sigv4_sign_request"
[2023-06-02T15:30:20Z DEBUG aws_smithy_http_tower::map_request] map_request; name="recursion_detection"
[2023-06-02T15:30:20Z DEBUG tracing::span] dispatch;
[2023-06-02T15:30:20Z DEBUG hyper::client::connect::dns] resolving host="<REDACTED>"
[2023-06-02T15:30:20Z DEBUG hyper::client::connect::http] connecting to <REDACTED>
[2023-06-02T15:30:20Z DEBUG hyper::client::connect::http] connected to <REDACTED>
[2023-06-02T15:30:20Z DEBUG rustls::client::hs] No cached session for DnsName(DnsName(DnsName("<REDACTED>")))
[2023-06-02T15:30:20Z DEBUG rustls::client::hs] Not resuming any session
[2023-06-02T15:30:20Z DEBUG rustls::client::hs] ALPN protocol is Some(b"http/1.1")
[2023-06-02T15:30:20Z DEBUG rustls::client::hs] Using ciphersuite TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
[2023-06-02T15:30:20Z DEBUG rustls::client::tls12] ECDHE curve is ECParameters { curve_type: NamedCurve, named_group: secp256r1 }
[2023-06-02T15:30:20Z DEBUG rustls::client::tls12] Server DNS name is DnsName(DnsName(DnsName("<REDACTED>")))
[2023-06-02T15:30:20Z DEBUG rustls::client::tls12] Session saved
[2023-06-02T15:30:20Z DEBUG hyper::proto::h1::io] flushed 686 bytes
[2023-06-02T15:30:20Z DEBUG hyper::proto::h1::io] parsed 10 headers
[2023-06-02T15:30:20Z DEBUG hyper::proto::h1::conn] incoming body is content-length (4746924 bytes)
[2023-06-02T15:30:20Z DEBUG tracing::span] load_response;
[2023-06-02T15:30:20Z DEBUG tracing::span] parse_unloaded;
[2023-06-02T15:30:20Z DEBUG aws_sdk_s3::operation::get_object] extended_request_id=Some(<REDACTED>)
[2023-06-02T15:30:20Z DEBUG aws_sdk_s3::operation::get_object] request_id=Some(<REDACTD>)
[2023-06-02T15:30:20Z DEBUG aws_smithy_client] send_operation; status="ok"
thread '<REDACTED>::bug_report' panicked at 'assertion failed: resp.last_modified.unwrap() > if_modified_since_input', <REDACTED>:389:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test <REDACTED>::bug_report ... FAILED

failures:

failures:
    <REDACTED>::bug_report

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 29 filtered out; finished in 0.27s

error: test failed, to rerun pass `--lib`

The text was updated successfully, but these errors were encountered:

Velfi · 2023-06-02T18:31:35Z

Hey @cowlicks, thanks for submitting this issue. We'll look into it.

Velfi · 2023-06-02T19:17:30Z

@cowlicks Ok, I figured it out. If you send dates with subsecond nanos, then S3 will silently ignore it. If you update your code above to this, then it should start working as expected:

#[tokio::test]
async fn bug_report() -> Result<()> {
    dotenvy::dotenv().ok();
    let config = ::aws_config::load_from_env().await;
    let client = aws_sdk_s3::Client::new(&config);
    let key = "my_key_that_exists.json";
    let bucket = "my-bucket";

    let if_modified_since_input: aws_sdk_s3::primitives::DateTime =
        std::time::SystemTime::now().into();
    if_modified_since_input.set_subsec_nanos(0);

    let resp = client
        .get_object()
        .bucket(bucket)
        .key(key)
        .if_modified_since(if_modified_since_input)
        .send()
        .await?;
    assert!(resp.last_modified.unwrap() > if_modified_since_input);
    dbg!(resp);
    Ok(())
}

@jmklix I would consider this a bug on S3's side. They should either accept datetimes with nanosecond precision or respond with a 400 explaining that they don't allow subsecond precision. Would you be able to check if other SDKs have this same issue and file a ticket with the S3 team?

Velfi · 2023-06-07T18:12:38Z

@jmklix I filed an issue with the S3 maintainers, ticket number is V925584723.

rcoh · 2023-08-08T17:41:16Z

It appears our date time formatting is incorrect:

ensure protocol tests exist in Smithy validating this behavior
fix our date time formatting
retest this API / Add protocol test specific to this API

rcoh · 2023-11-02T19:59:48Z

this is fixed in S3 0.35 (currently queued up to release)

github-actions · 2023-11-02T20:00:09Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

rcoh · 2023-11-07T17:01:43Z

Fix verified

cowlicks added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jun 2, 2023

Velfi removed the needs-triage This issue or PR still needs to be triaged. label Jun 2, 2023

rcoh added the p1 This is a high priority issue label Aug 8, 2023

rcoh mentioned this issue Sep 19, 2023

Omit fractional seconds from http-date format smithy-lang/smithy-rs#2989

Merged

1 task

rcoh added the pending-release This issue will be fixed by an approved PR that hasn't been released yet. label Oct 31, 2023

rcoh closed this as completed Nov 2, 2023

rcoh removed the pending-release This issue will be fixed by an approved PR that hasn't been released yet. label Nov 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`GetObjectFluentBuilder.if_modified_since` has no effect #818

`GetObjectFluentBuilder.if_modified_since` has no effect #818

cowlicks commented Jun 2, 2023 •

edited by Velfi

Loading

Velfi commented Jun 2, 2023

Velfi commented Jun 2, 2023

Velfi commented Jun 7, 2023

rcoh commented Aug 8, 2023 •

edited

Loading

rcoh commented Nov 2, 2023

github-actions bot commented Nov 2, 2023

rcoh commented Nov 7, 2023

GetObjectFluentBuilder.if_modified_since has no effect #818

GetObjectFluentBuilder.if_modified_since has no effect #818

Comments

cowlicks commented Jun 2, 2023 • edited by Velfi Loading

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

Version

Environment details (OS name and version, etc.)

Logs

Velfi commented Jun 2, 2023

Velfi commented Jun 2, 2023

Velfi commented Jun 7, 2023

rcoh commented Aug 8, 2023 • edited Loading

rcoh commented Nov 2, 2023

github-actions bot commented Nov 2, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

rcoh commented Nov 7, 2023

`GetObjectFluentBuilder.if_modified_since` has no effect #818

`GetObjectFluentBuilder.if_modified_since` has no effect #818

cowlicks commented Jun 2, 2023 •

edited by Velfi

Loading

rcoh commented Aug 8, 2023 •

edited

Loading