deprecate file_metadata stream #96
Conversation
cngpowered
changed the title
| - Verify no streams are synced when 'file_metadata' bookmark does not change | ||
| - Verify that the third sync with the updated simulated bookmark has the same synced streams as the first sync | ||
| - Verify that streams will sync based off of 'file_metadata' even when it is not selected |
There was a problem hiding this comment.
since file_metadata is now deprecated we don't rely on the file updated timestamp anymore, which was used as a pseudo bookmark earlier, however this does not impact as all the streams are full_table in nature.
removed the extra criteria from the test for the (now) deprecated stream
There was a problem hiding this comment.
In the PR description, it says you have added the exception handling, but looking at the code I don't see the changes anywhere.
Also can you please include the exact reason behind deprecating the file_metadata stream? Including this information would be helpful for future reference.
|
Also please remove the |
…-google-sheets into remove-filemetadata-stream
|
updated the PR details and removed references for file_metadata, also updated the tests. |
| "bookmarks": { | ||
| "file_metadata": "2019-09-27T22:34:39.000000Z" | ||
| } | ||
| "currently_syncing": "sheet_metadata" |
There was a problem hiding this comment.
Please include some stream name with the value. The state file looks incomplete.
There was a problem hiding this comment.
Hi @sgandhi1311 All the streams are essentially full table so we don't store date specific bookmarks for any stream.
|
Hi @sgandhi1311 and @Vi6hal , is there any plans on reintroducing something like this? We noticed our hourly syncs more than 10x the amount of data consumed each day. |
Description of change
file_metadataManual QA steps
Risks
Rollback steps
Why is file_metadata being removed ?
As per google's latest security updates on Oauth scopes, few scopes are determined as restricted thus requiring us to drop support for the file_metadata stream which utilizes one of the restricted scope.
Specifically: https://www.googleapis.com/auth/drive.metadata.readonly
Reference Document: https://developers.google.com/drive/api/guides/api-specific-auth#scopes