Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding s3_list_objects_encoding_type_url handler to ListObjectsV2 #1552

Merged
merged 6 commits into from
Sep 12, 2018
Merged

Adding s3_list_objects_encoding_type_url handler to ListObjectsV2 #1552

Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f9eef86
Adding s3_list_objects_encoding_type_url handler to ListObjectsV2
drocamor Sep 10, 2018
3231b9a
registering response decode for ListObjectsV2
drocamor Sep 11, 2018
8ef58cf
added decode_list_object_v2 handler and tests
drocamor Sep 11, 2018
a0d2f3f
generalized decode_list_object
drocamor Sep 12, 2018
321aece
list_objects_v2 decode integration test
drocamor Sep 12, 2018
9d31414
updating changelog
drocamor Sep 12, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions botocore/handlers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -880,6 +880,8 @@ def remove_subscribe_to_shard(class_attributes, **kwargs):

('before-parameter-build.s3.ListObjects',
set_list_objects_encoding_type_url),
('before-parameter-build.s3.ListObjectsV2',
set_list_objects_encoding_type_url),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also need to register the decoding after the response is received (line 945).

One handler injects the encoding type (set_list_objects_encoding_type_url), and the other handler (decode_list_object) decodes objects if we injected the encoding type param. We need both.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah got it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if we can add an integration test for this. Honestly, I would recommend adding one similar to the one that added the customization: https://github.com/boto/botocore/pull/726/files#diff-02f637532aa2fba41fa91ac0763345b4

('before-call.s3.PutBucketTagging', calculate_md5),
('before-call.s3.PutBucketLifecycle', calculate_md5),
('before-call.s3.PutBucketLifecycleConfiguration', calculate_md5),
Expand Down Expand Up @@ -943,6 +945,7 @@ def remove_subscribe_to_shard(class_attributes, **kwargs):
('before-parameter-build.route53', fix_route53_ids),
('before-parameter-build.glacier', inject_account_id),
('after-call.s3.ListObjects', decode_list_object),
('after-call.s3.ListObjectsV2', decode_list_object),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So unfortunately. We cannot purely reuse the decode_list_objects. There are different keys that need to be decoded according to the docs. Specifically, we need to decode: Delimiter, Prefix, ContinuationToken, Key, and StartAfter. I would recommend trying to see if we can consolidate the logic and make a new decode_list_objects_v2 handler that decodes it for the V2 response shape and use that instead.


# Cloudsearchdomain search operation will be sent by HTTP POST
('request-created.cloudsearchdomain.Search',
Expand Down