S3 object XML parsing error for large files

[bhaan]

Bug Report

Version

├── aws-sdk-s3 v0.0.16-alpha (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9)
│   ├── aws-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9) (*)
│   ├── aws-endpoint v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9) (*)
│   ├── aws-http v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9) (*)
│   ├── aws-hyper v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9) (*)
│   ├── aws-sig-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9) (*)
│   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9) (*)
│   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9) (*)
│   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9) (*)
│   └── smithy-xml v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.16-alpha#dab3b0a9)

Platform

Linux DELL-XPS-13-9380 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

AWS Services

S3

Description

The list_objects client function produces a XML parsing error for large files because the model::Object has a size (in bytes) with type i32, which I believe maxes out around 2GB.
https://github.com/awslabs/aws-sdk-rust/blob/main/sdk/s3/src/model.rs#L11230-L11231
https://github.com/awslabs/aws-sdk-rust/blob/main/sdk/s3/src/xml_deser.rs#L3958-L3961

Example code:

let client = aws_sdk_s3::Client::from_env();
let result = client
    .list_objects()
    .bucket(&bucket)
    .prefix(&key)
    .send()
    .await;

I expected an Ok result

Instead I got the following ListObjectsError:

ServiceError { err: ListObjectsError { kind: Unhandled(Custom("expected (integer: `com.amazonaws.s3#Size`)")) ...

With the following relevant trace logs:

[2021-08-27T20:27:54Z TRACE smithy_http_tower::dispatch] request=Request { method: GET, uri: REDACTED, version: HTTP/1.1, headers: {"content-type": "application/xml", "content-length": "0", "host": "s3.us-east-1.amazonaws.com", "user-agent": "aws-sdk-rust/0.1.0 os/linux lang/rust/1.53.0", "x-amz-user-agent": "aws-sdk-rust/0.1.0 api/s3/0.0.16-alpha os/linux lang/rust/1.53.0", "authorization": Sensitive, "x-amz-date": "20210827T202754Z", "x-amz-content-sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}, body: SdkBody { inner: Once(Some(b"")), retryable: true } } 
[2021-08-27T20:27:54Z TRACE smithy_http::middleware] http_response=Response { status: 200, version: HTTP/1.1, headers: {"x-amz-id-2": "jXU2ZMxiZIVhIp0Dq+xJ82p+vnsHmTwdFFdAGjexwnghDvmkx7QWeM4+qn6opKZxM3Vkf2aG010=", "x-amz-request-id": "HDC393J1HSBX3PVY", "date": "Fri, 27 Aug 2021 20:27:55 GMT", "x-amz-bucket-region": "us-east-1", "content-type": "application/xml", "transfer-encoding": "chunked", "server": "AmazonS3"}, body: b"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<ListBucketResult xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"><Name>REDACTED</Name><Prefix>REDACTED</Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>REDACTED</Key><LastModified>2021-08-26T23:00:01.000Z</LastModified><ETag>&quot;fd71cdbe67710902513479d9650b0785&quot;</ETag><Size>2940645877</Size><Owner><ID>REDACTED</ID><DisplayName>REDACTED</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents></ListBucketResult>" } 

Notice the part <Size>2940645877</Size>, which is greater than the max value of i32: 2147483647

[rcoh]

looks like a model issue. Thanks for the report, we'll get this fixed!

[jdisanti]

This has been fixed in v0.0.17-alpha. Thanks!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.