Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(storage-browser): listCallerAccessGrantsDeserializer not parsing multiple AccessGrant instances #13671

Conversation

ashwinkumar6
Copy link
Member

Description of changes

  • The listCallerAccessGrantsDeserializer does not handle parsing multiple AccessGrantsInstance.
  • When the XML response from service contains a single AccessGrantsInstance items it's parsed into an object; while it's parsed into an array if there are multiple items

Output log before fix
xml parsed

// 1 item
{
  "CallerAccessGrantsList": {
    "AccessGrantsInstance": {
      "ApplicationArn": "ALL",
      "Permission": "READWRITE",
      "GrantScope": "s3://ashwin-portcullis-test1/*"
    }
  }
}

// 2 items
{
  "CallerAccessGrantsList": {
    "AccessGrantsInstance": [
      {
        "ApplicationArn": "ALL",
        "Permission": "READWRITE",
        "GrantScope": "s3://ashwin-portcullis-test1/*"
      },
      {
        "ApplicationArn": "ALL",
        "Permission": "READWRITE",
        "GrantScope": "s3://ashwin-portcullis-test2/*"
      }
    ]
  }
}

Deserializer output

// 1 item
{
  "CallerAccessGrantsList": [
    {
      "ApplicationArn": "ALL",
      "GrantScope": "s3://ashwin-portcullis-test1/*",
      "Permission": "READWRITE"
    }
  ]
}
// 2 items
{
  "CallerAccessGrantsList": [
    {} ====> ERROR
  ]
}

Issue #, if available

Description of how you validated changes

  • Updated unit tests
  • linked lib and tested locally with AG setup in backend

Checklist

  • PR description included
  • yarn test passes
  • Unit Tests are changed or added
  • Relevant documentation is changed or added (and PR referenced)

Checklist for repo maintainers

  • Verify E2E tests for existing workflows are working as expected or add E2E tests for newly added workflows
  • New source file paths included in this PR have been added to CODEOWNERS, if appropriate

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ashwinkumar6 ashwinkumar6 requested a review from a team as a code owner August 1, 2024 20:24
@ashwinkumar6 ashwinkumar6 changed the title fix(storage-browser): listCallerAccessGrantsDeserializernot parsing multiple AccessGrant instances fix(storage-browser): listCallerAccessGrantsDeserializer not parsing multiple AccessGrant instances Aug 1, 2024
@@ -54,6 +54,11 @@ const listCallerAccessGrantsHappyCase: ApiFunctionalTestCase<
<GrantScope>${MOCK_GRANT_SCOPE}</GrantScope>
<Permission>${MOCK_PERMISSION}</Permission>
</AccessGrantsInstance>
<AccessGrantsInstance>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: don't we also need to test and verify the case of single AccessGrantsInstance?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good callout, can we add an additional test case please?

Copy link
Member Author

@ashwinkumar6 ashwinkumar6 Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx updated,

Considering the fact that the default behavior of xml -> obj parsing is

  • single item = object
  • multiple items = array of object

The underlying parseXmlBody util is used across all storage APIs. Should we add tests for 'single and multiple xml items' test in other places as well (example ListObjectV2 only has multiple items)
(probably in another PR if required)

Copy link
Member

@calebpollman calebpollman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@ashwinkumar6 ashwinkumar6 merged commit 08ef01b into aws-amplify:storage-browser/main Aug 1, 2024
28 checks passed
@ashwinkumar6 ashwinkumar6 deleted the storage-browser/main branch August 1, 2024 22:51
ashika112 added a commit that referenced this pull request Nov 20, 2024
* chore: enable storage-browser preid release

* chore: sync main (#13478)

* release(required): Parsing custom oAuth in amplify_outputs (#13474)

* update parseAmplify logic

* revert custom oAuth from gen1 config

* update bundle size

* chore(release): Publish [skip release]

 - @aws-amplify/adapter-nextjs@1.2.3
 - @aws-amplify/analytics@7.0.35
 - @aws-amplify/api@6.0.37
 - @aws-amplify/api-graphql@4.1.6
 - @aws-amplify/api-rest@4.0.35
 - @aws-amplify/auth@6.3.5
 - aws-amplify@6.3.6
 - @aws-amplify/core@6.3.2
 - @aws-amplify/datastore@5.0.37
 - @aws-amplify/datastore-storage-adapter@2.1.37
 - @aws-amplify/geo@3.0.35
 - @aws-amplify/interactions@6.0.34
 - @aws-amplify/notifications@2.0.35
 - @aws-amplify/predictions@6.1.10
 - @aws-amplify/pubsub@6.1.9
 - @aws-amplify/storage@6.4.6
 - tsc-compliance-test@0.1.39

* chore(release): update API docs [skip release]

---------

Co-authored-by: ashika112 <155593080+ashika112@users.noreply.github.com>
Co-authored-by: aws-amplify-bot <aws@amazon.com>
Co-authored-by: Jim Blanchard <jim.l.blanchard@gmail.com>

* feat(storage): add delimiter support (#13480)

* feat: add delimiter input/output types

* feat: pass Delimiter parameter to ListObjectsV2 API

* chore: add unit tests

* chore: bump bunde size

* chore: address feedback

* chore: fix build

* chore: address feedback

* chore: address feedback

* chore: address feedback

* chore: enable storage-browser preid release (#13524)

chore: fix npm dist tag to be storage-browser

* feat(storage): add base types for storage browser (#13528)

* feat(storage): add base types

* fix(storage): address feedbacks

Co-authored-by: israx <70438514+israx@users.noreply.github.com>

* feat(storage): add creds store scaffolding and update types (#13558)

* feat(storage): add cred store lru implementation (#13561)

* refactor(storage): decouple utils from Amplify singleton (#13562)

* feat: add config constructor

* refactor: remove singleton reference from storage utils

* refactor: update storage utils

* chore: update upload api

* chore: address feedback

* chore: fix unit tests

* chore: remove singleton reference

* chore: add license headers

* chore: address feedback

* chore: update bundle size

* chore: address feedback

* chore: update bundle size

* feat(storage): add cred store creation implementation (#13575)

Co-authored-by: israx <70438514+israx@users.noreply.github.com>

* feat(storage): use WeakMap for store registry (#13586)

* feat(storage): use WeakMap for store registry

* chore(storage): export storage browser utils from @aws-amplify/storage/storage-browser

* doc(storage): add disclaimer

* feat(storage): Added getDataAccess & listCallerAccessGrant clients (#13582)

* feat(storage): add an adapter interface for storage browser (#13576)


---------

Co-authored-by: Jim Blanchard <jim.l.blanchard@gmail.com>

* Revert "refactor(storage): decouple utils from Amplify singleton (#13562)" (#13597)

This reverts commit 1079e7f.

* feat(storage): simplify the location cred provider option input (#13601)

Remove the un-essetnial validation of per-API's location credentials provider
input scope and permission for now.

* feat: Implement getLocationCredentials handler & integrate with adapter (#13600)

* feat(storage): implement listLocations API and creation handler (#13602)

* chore: expose path storage-browser from scoped package (#13611)

chore: expose path storage-browser from scoped package

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* feat(storage): enables location credentials provider (#13605)

* feat: add location credentials provider

* chore: add unit tests

* chore: address feedback

* chore: add locationCredentialsOption to copy

* chore: remove casting types

* chore: assert idenitity id

* chore: avoid export common options interface

* chore: address feedback

* chore: fix test

* chore: address feedback

* address feedback

* chore: clean-up types

* chore: add test

* chore: update api bundlesize

* feat(storage): resolve merge issue with multibucket

* chore: update bundle size for config change and s3 multibucket

* chore: address feedbacks

* feat(storage): introduce preventOverwrite option to uploadData via HeadObject (#13640)

* feat(storage): introduce preventOverwrite operation to uploadData via HeadObject

* fix: add missing license and remove dependency on core in preventOverwrite validator

* chore: update storage:uploadData bundle size

* feat: move existing object validation to before completeMultipartUpload

* fix: increase storage:uploadData bundle size

* fix(storage): export storage-browser types for TS v4.2+ (#13647)

* chore(storage-browser): export store and credentials related types, update createListLocationsHandler (#13660)

* feat(storage): support force refresh location credentials (#13589)

* feat(storage): require temporary creds for storage browser interfaces (#13664)

* feat: introduce CRC32 checksums to storage:uploadData API (#13649)

Co-authored-by: Donny Wu <wuuxi@amazon.com>

* fix(storage-browser): listCallerAccessGrantsDeserializer not parsing multiple AccessGrant instances (#13671)

* fix(storage-browser): listCallerAccessGrantsDeserializernot parsing multiple AccessGrant instances

* chore: add unit tests for single and multiple grants

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore(storage-browser): expose additional input output types (#13682)

* chore(storage-browser): expose additional internal types

* address feedback

* remove 'applicationArn' from listCallerAccessGrant unit test

* Update packages/storage/src/storageBrowser/apis/listCallerAccessGrants.ts

Co-authored-by: Caleb Pollman <cpollman1@gmail.com>

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>
Co-authored-by: Caleb Pollman <cpollman1@gmail.com>

* chore: add ui to storage browser co-owner

* chore: update bundle size

* chore: enable storage-browser integ test (#13698)

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore(storage): update s3 control model (#13705)

* chore(storage): update s3 control model

* fix: move permission validation to custom client

* chore: formatting code

* chore(storage-browser): pin crc-32 dep at 1.2.2 (#13752)

* chore(storage-browser): pin crc-32 dep at 1.2.2

* chore: update lock file

* chore(storage): add durability check for urls in put object (#13746)

chore: add durability check for presigned urls in put object

* fix(storage-browser): missing error wrapping for s3 control responses (#13779)

* feat: validate corroborating elements in response, and compare echoed elements between request/response (#13764)

* feat: add list call response validation

* fix: add input key to test mock

* feat: align list response validations with design

* fix: increment bundle size for storage:list

---------

Co-authored-by: Donny Wu <wuuxi@amazon.com>

* feat(storage): add support for conditional headers to copy, and validate serialization (#13772)

* feat: add notModifiedSince and eTag to copy

* missing tests for durability helpers

* add tests for integrity helpers

* feat: add URL validation and tests for copyObject

* chore: increase bundle size of storage:copy

* feat: clean-up copy header validation logic

* fix: revert copy option interface name changes

---------

Co-authored-by: Donny Wu <wuuxi@amazon.com>

* feat: adding object url checks (#13810)

* chore: durability checks for create & complete multipart (#13809)

* chore: durability checks for create & complete multipart

* fix: parsePayload mock path

* chore: enable e2e and tagged release (#13848)

* chore: enable e2e and tagged release

* core: update tag name

* Revert "core: update tag name"

This reverts commit c994f51.

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore: Setup storage internals route (#13858)

* chore: increse bundle size

* chore: Refactor contents of `storage-browser/` path into `internals/` (#13859)

* feat(storage): export Default Part Size constant (#13851)

* chore: adds a flag to uploadData api output

* chore: update comments for the flag

* chore: revert some naming changes

* chore: revert and make internal paramter optional with a default value

* chore: update bundle size

* chore: update to expose constant instead of bool in return type

* chore: update export tests

* chore: increase bundle size

* chore: Setup storage internals route (#13858)

* chore: Refactor contents of `storage-browser/` path into `internals/` (#13859)

* fix: multipart upload storage crc32 integrity (#13878)

* fix: multipart upload storage crc32 integrity

* address comments

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* fix linter

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>
Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* feat(storage): internal GetProperties API (#13869)


---------

Co-authored-by: Jim Blanchard <jim.l.blanchard@gmail.com>

* feat(storage): add new internal remove api (#13880)

feat(internal-remove): add new internal remove api

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore: Add internal getUrl API (#13882)

* feat(storage): internals list API  (#13874)

* feat(storage): add new internal downloadData api (#13887)

* feat(internal-remove): add new internal downloadData api

* code cleanup

* code cleanup

* chore: fix ts doc

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* feat: optional checksum algorithm for upload (#13849)

* feat: opt in checksum

* fix: revert local prettier suggestion

* fix: up size limit for storage upload data

* feat: react native crc32

* fix: up bundle size limit and fix typo

* feat: add documentation for checksumAlgorithm

* fix: update bundle size limit

* fix: update bundle size limit

* fix: address pr feedbacks

* fix: bundle-size limit

---------

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* feat(storage): add remaining copy changes for internals (#13889)

* chore: add remaining copy changes for internals

* chore; explicitly call internal API with params

* chore: copy return type

* chore: feedback changes

* feat(storage): internal uploadData implementation (#13888)

* feat(storage): internal uploadData implementation

* chore: update bundle size

* fix: address feedbacks

* revert: optional checksum algorithm for upload (#13849) (#13910)

This reverts commit 02cb08a.

* fix(storage): internals list function not able to decide which output types to use (#13915)

chore: add function overloading to list

* feat(storage): Add API support for Expected Bucket Owner  (#13914)

* Update Top/Internal API for expected bucket owner feat

---------

Co-authored-by: JoonWon Choi <joonwonc@amazon.com>

* chore: sync main with storage browser integrity branch (#13928)

* chore(release): Set core metadata [skip release]

* chore(release): Publish [skip release]

 - @aws-amplify/adapter-nextjs@1.2.21
 - @aws-amplify/analytics@7.0.51
 - @aws-amplify/api@6.0.53
 - @aws-amplify/api-graphql@4.4.0
 - @aws-amplify/api-rest@4.0.51
 - @aws-amplify/auth@6.5.1
 - aws-amplify@6.6.4
 - @aws-amplify/core@6.4.4
 - @aws-amplify/datastore@5.0.53
 - @aws-amplify/datastore-storage-adapter@2.1.53
 - @aws-amplify/geo@3.0.51
 - @aws-amplify/interactions@6.0.50
 - @aws-amplify/notifications@2.0.51
 - @aws-amplify/predictions@6.1.26
 - @aws-amplify/pubsub@6.1.26
 - @aws-amplify/storage@6.6.9
 - tsc-compliance-test@0.1.56

* chore(release): Update API docs [skip release]

* chore: update contributing guide vite troubleshooting (#13826)

* docs: update contributing guide vite troubleshooting

* Update CONTRIBUTING.md

---------

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* chore(ai): add UpdateConversation enum (#13920)

* chore(release): Set core metadata [skip release]

* chore(release): Publish [skip release]

 - @aws-amplify/adapter-nextjs@1.2.22
 - @aws-amplify/analytics@7.0.52
 - @aws-amplify/api@6.0.54
 - @aws-amplify/api-graphql@4.4.1
 - @aws-amplify/api-rest@4.0.52
 - @aws-amplify/auth@6.5.2
 - aws-amplify@6.6.5
 - @aws-amplify/core@6.4.5
 - @aws-amplify/datastore@5.0.54
 - @aws-amplify/datastore-storage-adapter@2.1.54
 - @aws-amplify/geo@3.0.52
 - @aws-amplify/interactions@6.0.51
 - @aws-amplify/notifications@2.0.52
 - @aws-amplify/predictions@6.1.27
 - @aws-amplify/pubsub@6.1.27
 - @aws-amplify/storage@6.6.10
 - tsc-compliance-test@0.1.57

* chore(release): Update API docs [skip release]

* fix(storage): multipart upload is firing on 0 bytes data (#13927)

* chore: update bundle size

---------

Co-authored-by: erinleigh90 <106691284+erinleigh90@users.noreply.github.com>
Co-authored-by: aws-amplify-bot <aws@amazon.com>
Co-authored-by: Parker Scanlon <69879391+scanlonp@users.noreply.github.com>
Co-authored-by: AllanZhengYP <zheallan@amazon.com>
Co-authored-by: Danny Banks <djb@amazon.com>
Co-authored-by: ashika112 <155593080+ashika112@users.noreply.github.com>
Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com>

* feat: Add expectedBucketOwner to remaining internal APIs (#13932)

feat: Add expectedBucketOwner to remaining internal APIs.

* feat(storage): add advanced option to disable upload cache (#13931)

* feat(storage): set allowedByApp to listCallerAccessGrants (#13934)

* Storage Browser Default Auth (#13866)

* first draft poc

* upadtes

* add listPaths API

* update new file structure

* fix types

* refactor types and utils

* update tests

* fix test

* fix bundle size test

* update the listLocation handler

* rename util

* update Path type

* fix missed type

* chore(ci): add new storage-gen2-internal e2e test (#13916)

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore(storage): remove credential store and managed auth config factory (#13944)

* feat: optional checksum algorithm for upload (#13939)

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* feat: preventOverwrite with if-none-match header (#13954)

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* Feat: Add pagination to Amplify Default Auth storage Browser (#13897)

* update the listLocation handler
* implement memoization
* add pagination logic
* update usergroup logic & test
* update getPaginated Locations
* fix failing test

* test(storage): refactor unit tests for public & internal facade (#13955)

* Chore: Remove createAmplifyAdapter & refactor (#13974)

remove createAmplifyAdapter & refactor

* update bundle size

* fix bundle size & test

* Fix: retry failure in storage retryDecider (#13977)

fix retry failure

* feat: validate uploaded parts before completing upload (#13763)

Co-authored-by: Donny Wu <wuuxi@amazon.com>
Co-authored-by: Allan Zheng <zheallan@amazon.com>

* feat(storage): add customEndpoint to internal apis in advanced options (#13961)

* feat: add baseEndpoint to advanced options

* feat: add baseEndpoint to customEndpoint

* feat: thread baseEndpoint through resolved config to endpoint resolver

* add customEndpoint advanced option to internals storage data-plane apis

* add customEndpoint advanced option to internals storage control-plane apis

* fix unit test

* code cleanup

* increase bundle size

* wire up customEndpoint on copy API

* increase the bundle size

* add customEndpoint unit tests for all data and control apis

* increase bundle size

* update ts docs

* add additional error unit tests for endpointResolver

* add unit tests for internals/ apis

* code cleanup

* address feedback

* add comment for ForcePathStyleEndpointNotSupported ErrorCode

* increase bundle size

* remove docs links from error recovery message

---------

Co-authored-by: Erin Beal <erinleig@amazon.com>
Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* fix(core): support endpoint resolver accepting both input and config(#13985)

* feat(storage): allow checksum algo for internal upload API (#14002)

* feat: allow setting encoding type for list calls

* refactor: move to list internal

* fix: add license

* fix(storage): bug in copy unmodified since to use UTC (#14025)

fix to use UTC and test

* Revert "chore: enable e2e and tagged release (#13848)"

This reverts commit 3dfde04.

* revert tagged release

---------

Co-authored-by: Allan Zheng <zheallan@amazon.com>
Co-authored-by: israx <70438514+israx@users.noreply.github.com>
Co-authored-by: ashika112 <155593080+ashika112@users.noreply.github.com>
Co-authored-by: aws-amplify-bot <aws@amazon.com>
Co-authored-by: Jim Blanchard <jim.l.blanchard@gmail.com>
Co-authored-by: Ashwin Kumar <ashwinkumar2468@gmail.com>
Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>
Co-authored-by: Jamie Epp <168486127+eppjame@users.noreply.github.com>
Co-authored-by: Jamie Epp <eppjame@amazon.com>
Co-authored-by: Caleb Pollman <cpollman@amazon.com>
Co-authored-by: Donny Wu <wuuxi@amazon.com>
Co-authored-by: Caleb Pollman <cpollman1@gmail.com>
Co-authored-by: Sergio Castillo Yrizales <yriz@amazon.com>
Co-authored-by: Joon Choi <requiemdeciel@gmail.com>
Co-authored-by: JoonWon Choi <joonwonc@amazon.com>
Co-authored-by: erinleigh90 <106691284+erinleigh90@users.noreply.github.com>
Co-authored-by: Parker Scanlon <69879391+scanlonp@users.noreply.github.com>
Co-authored-by: Danny Banks <djb@amazon.com>
Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com>
Co-authored-by: ashika112 <akasivis@amazon.com>
Co-authored-by: Erin Beal <erinleig@amazon.com>
Co-authored-by: Pranav Malewadkar <pranavml@amazon.com>
Co-authored-by: Pranav Malewadkar <malewadkar.1@osu.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants