Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Blob][STG73]Blob Tags #11418

Merged
merged 7 commits into from
Jun 10, 2020
Merged

[Blob][STG73]Blob Tags #11418

merged 7 commits into from
Jun 10, 2020

Conversation

xiafu-msft
Copy link
Contributor

No description provided.

@xiafu-msft xiafu-msft marked this pull request as ready for review May 14, 2020 00:23
xiafu-msft
xiafu-msft commented May 14, 2020
View reviewed changes
sdk/storage/azure-storage-blob/azure/storage/blob/_blob_service_client.py Outdated
@@ -419,28 +419,20 @@ def list_containers(
@distributed_trace
def filter_blobs(self, where=None, **kwargs):
# type: (Optional[str], Optional[Any], **Any) -> ItemPaged[BlobProperties]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ItemPaged[FilteredBlob]

annatisch
annatisch reviewed May 14, 2020
View reviewed changes
sdk/storage/azure-storage-blob/azure/storage/blob/_blob_service_client.py Outdated
@@ -416,6 +416,36 @@ def list_containers(
page_iterator_class=ContainerPropertiesPaged
)

@distributed_trace
def filter_blobs(self, where=None, **kwargs):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm let me think about this name....
We're calling it this across all languages?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a convention to start APIs returning ItemPaged to start with list_ - i know list_blobs exists - but still a little unconvinced with the name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about find_blobs_by_tags? Java is using this name

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does it mean if where=None? Is that the same behaivour as "list all blobs in all containers"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to filter_expression

annatisch
annatisch reviewed May 14, 2020
View reviewed changes
sdk/storage/azure-storage-blob/tests/test_blob_tags_async.py Outdated
@AsyncStorageTestCase.await_prepared_test
async def test_create_append_blob_with_tags(self, resource_group, location, storage_account, storage_account_key):
await self._setup(storage_account, storage_account_key)
blob_tags = {"+-./:=_ ": "firsttag", "tag2": "+-./:=_", "+-./:=_1": "+-./:=_"}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to test case sensitivity?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's supposed to be case sensitive, I updated the doc. While the service fix hasn't reached to the test tenant...So we cannot have an accurate client side test currently.

annatisch
annatisch requested changes May 14, 2020
View reviewed changes
Copy link
Member

@annatisch annatisch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments listed above :)

@annatisch annatisch added APIChange This PR contains an addition or change to the API signature and must be reviewed by an architect. Storage Storage Service (Queues, Blobs, Files) labels May 15, 2020
annatisch
annatisch reviewed May 29, 2020
View reviewed changes
sdk/storage/azure-storage-blob/azure/storage/blob/_serialize.py
if components:
del components[-1]

return ''.join(components)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use '&'.join(components) then you wont have to delete it off the end in the line above :)

Copy link
Contributor Author

@xiafu-msft xiafu-msft Jun 4, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currently the component list is like ["key", "=", "value", "&", "key2", "=", "value2", "&" ], so I "&" wasn't used to concatenate those items. While we can make it ["key=value","key2=value2"] and join using "&"
I guess we don't want to concatenate the key value into "key=value" first? (Just my thought)

@xiafu-msft xiafu-msft merged this pull request into Azure:feature/storage-stg73 Jun 10, 2020
xiafu-msft added a commit that referenced this pull request Jun 10, 2020
@xiafu-msft
[Blob][STG73]Blob Tags (#11418)
60ada0b
xiafu-msft added a commit that referenced this pull request Jul 7, 2020
@xiafu-msft @kasobol-msft
@zezha-msft @annatisch
Feature/storage stg73 (#12272)
900aa7d 
* [Storage]STG73

* [Blob][Swagger]Update Swagger (#10943)

* [Blob][Swagger]Regenerate Swagger Code

* fix container test failure caused by list_containers include type change

* [Storage] Internal avro parser. (#10764)

* initial avro parser

* try fixing test...

* falling in love with python compatibility...

* make linter happy.

* raise StopIteration when there is no more bytes instead of tracking file length.

* async avro parser

* fix syntax for Python 3.5

* get rid of 'readers_schema' as we only honor schema that has been written to file ('writer_schema').

* pr feedback

* trim unused code.

* pr feedback.

* simplify skip sync in next.

* move avro tests from _shared.

* Jumbo blob support (#11176)

* wip

* initial test coverage.

* wip.

* wip

* single upload.

* add async tests.

* disable 50k block tests.

* datalake append.

* async datalake

* disable tests that send large payload over network.

* pr feedback.

* Undelete share (#11394)

* Undelete container (#11339)

* [Storage][Blob] Added support for Object Replication (#11525)

* Blob versioning (#11154)

* [Blob][QuickQuery]Add Quick Query Support (#10946)

* [Blob][STG73]Blob Tags (#11418)

* regenerate code (#11964)

* fix the bug which caused only showing fatal error (#11997)

* [Storage][STG73]Address API Review Comments (#12111)

* [Storage][STG73]Address API Review Comments

* [Storage][STG73]dict<policy, rules> -> list(ObjectReplicationPolicy)

* fix blob tag_value test

* expose ObjectReplicationPolicy and ObjectReplicationRule, fix test

* fix test

* Changefeed (#10755)

* [ChangeFeed]Add ChangeFeed Package

* test_avro failure

* update dev_requirement.txt

* change namespace to azure.storage.blob.changefeed

* address comments

* optimize memory when reading changefeed events

* namespace change

* set up package change

* fix failed tests

* readme and kwargs

* Update sdk/storage/azure-storage-blob-changefeed/azure/storage/blob/changefeed/_change_feed_client.py

Co-authored-by: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>

* address comments

* 'azure-storage-blob>=12.3.0' which does not match the frozen requirement 'azure-storage-blob~=1.3'

Co-authored-by: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>

* [Storage-Blob] Quick Query API (#11991)

* Renamed query error

* Renamed query reader

* Updated config models

* Updated format request params

* Updated iterator

* fix the bug which caused only showing fatal error

* Updated Error message

* Fixed query helper

* Started test conversion

* small fix

* Fixed tests

* Updated error handling + json model

* Updated recordings

* Removed old recording

* Added iter tests

* Iter test recordings

* Fix test

* Remove extra recording

* Fix pylint

* Some docs cleanup

* Renamed iter_records -> iter_stream

* Review feedback

* Updated tests

* Missing commas

* Fix syntax

* Fix pylint

Co-authored-by: xiafu <xiafu@microsoft.com>

* tag sas (#12258)

* tag sas

* disable undelete_container

* pylint

* skip undelete_container tests

* [Blob][Versioning]Disable Versioning Live Test (#12281)

* [Blob][QQ]Default output_format to input_format (#12283)

* [Storage][Jumbo]Remove super (#12314)

* [Storage][JumboBlob]remove empty super()

* pypy3

* change sas version to latest

* set tags account location to central canada

* re-recording queue

* changefeed paths generator

* mark tests for vid as playback only

* fix changefeed

* fix pylint
make the test account location for tags to central canada

* add a delay before calling find_blobs_by_tags

* remove tags header

* mark a large file test playback only

* revert "mark a large file test playback only"
skip upload large file test
address comment

* move tag permission and filter_by_tags permission to kwargs

Co-authored-by: Kamil Sobol <61715331+kasobol-msft@users.noreply.github.com>
Co-authored-by: Ze Qian Zhang <zezha@microsoft.com>
Co-authored-by: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>
Co-authored-by: annatisch <antisch@microsoft.com>
@mohsha-msft mohsha-msft mentioned this pull request Sep 24, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amishra-dev amishra-dev Awaiting requested review from amishra-dev

@zezha-msft zezha-msft Awaiting requested review from zezha-msft

@annatisch annatisch Awaiting requested review from annatisch

@rakshith91 rakshith91 Awaiting requested review from rakshith91

Assignees
No one assigned
Labels
APIChange This PR contains an addition or change to the API signature and must be reviewed by an architect. Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@xiafu-msft @annatisch @rakshith91