Skip to content

Commit

Permalink
[AIRFLOW-4255] Replace Discovery based api with client based for GCS (a…
Browse files Browse the repository at this point in the history
  • Loading branch information
kaxil authored and wayne.morris committed Jul 29, 2019
1 parent 908722f commit 3e648b7
Show file tree
Hide file tree
Showing 6 changed files with 334 additions and 639 deletions.
22 changes: 22 additions & 0 deletions UPDATING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,28 @@ assists users migrating to a new version.

## Airflow Master

### Changes to GoogleCloudStorageHook

* the discovery-based api (`googleapiclient.discovery`) used in `GoogleCloudStorageHook` is now replaced by the recommended client based api (`google-cloud-storage`). To know the difference between both the libraries, read https://cloud.google.com/apis/docs/client-libraries-explained. PR: [#5054](https://github.com/apache/airflow/pull/5054)
* as a part of this replacement, the `multipart` & `num_retries` parameters for `GoogleCloudStorageHook.upload` method has been removed:

**Old**:
```python
def upload(self, bucket, object, filename,
mime_type='application/octet-stream', gzip=False,
multipart=False, num_retries=0):
```

**New**:
```python
def upload(self, bucket, object, filename,
mime_type='application/octet-stream', gzip=False):
```

The client library uses multipart upload automatically if the object/blob size is more than 8 MB - [source code](https://github.com/googleapis/google-cloud-python/blob/11c543ce7dd1d804688163bc7895cf592feb445f/storage/google/cloud/storage/blob.py#L989-L997).

* the `generation` parameter is no longer supported in `GoogleCloudStorageHook.delete` and `GoogleCloudStorageHook.insert_object_acl`.

### Changes to CloudantHook

* upgraded cloudant version from `>=0.5.9,<2.0` to `>=2.0`
Expand Down
Loading

0 comments on commit 3e648b7

Please sign in to comment.