Fix S3 ListParts pagination infinite loop #3679
Conversation
landonxjames
changed the title
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
Adding new trait IsTruncatedPaginatorTrait that indicates the Paginator should use the value of the isTruncated field to indicate if the pagination has been exhausted.
landonxjames
force-pushed
the
landonxjames/list-parts-page
branch
from
d6b8de1 to
40a0381
Compare
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
landonxjames
force-pushed
the
landonxjames/list-parts-page
branch
from
62d4f62 to
3d73844
Compare
Fixing up isTruncatedPaginator model test
landonxjames
force-pushed
the
landonxjames/list-parts-page
branch
from
3d73844 to
901144e
Compare
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/customize/s3/S3Decorator.kt
Outdated
Show resolved
Hide resolved
| rustTemplate( | ||
| """ | ||
| // Pagination is exhausted when `is_truncated` is false | ||
| let is_empty = !resp.is_truncated.unwrap_or(false); |
There was a problem hiding this comment.
nit: This assumes the output shape member name is is_truncated which while likely true we could probably be more precise by having the trait include the member shape ID of the truncation member.
| * In this case we use a false value of isTruncated as the only indicator that | ||
| * the pagination is exhausted. | ||
| * */ | ||
| private fun isEmptySetter() = |
There was a problem hiding this comment.
If you wanted to go even more down the rabbit hole you could declare a section for this bit of code and then register a customization for that section that does something different. You would then basically be able to move all of this into a single TruncatedPaginatorDecorator that handles both the model transform and the actual codegen logic.
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
Merging this for now to make sure it goes in with the next smithy-rs release since it fixes an actual customer bug. Going to connect with @aajtodd on Monday about the unaddressed suggestions to hopefully get those in as well. |
Motivation and Context
The paginator for the S3
ListPartsoperation could loop forever: awslabs/aws-sdk-rust#1143. This is because most paginated operations use an empty next token as an indication that pagination is exhausted.ListPartsinstead sets the final next token as0causing the pagination to loop back to the first page and loop forever. Instead of an empty next tokenListPartsusesIsTruncated = falseto indicate that pagination has been exhausted.Description
isTruncatedPaginatorTraitListPartsOutputshapeis_emptyvalue in the paginator.is_truncatedfieldTesting
is_truncated = falseNote: I'm still working on turning this into a model test rather than an S3 specific integration test, but I wanted to get some feedback on the actual fix while I'm figuring that out)
Checklist
CHANGELOG.next.tomlif I made changes to the smithy-rs codegen or runtime cratesCHANGELOG.next.tomlif I made changes to the AWS SDK, generated SDK code, or SDK runtime cratesBy submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.