Skip to content

Commit

Permalink
Docs: AWS S3 Data Reading (#1063)
Browse files Browse the repository at this point in the history
* add S3 AWS docs for data reading

* line wrap fix

* formatting

* setting

* boto3 ref

* update ref

* ref
  • Loading branch information
taylorfturner authored Nov 13, 2023
1 parent cc25a9b commit ec7fc9b
Showing 1 changed file with 19 additions and 1 deletion.
20 changes: 19 additions & 1 deletion docs/source/data_readers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -161,4 +161,22 @@ URL may also be specified using the options dict parameter.

Possible `options`:

* verify_ssl: must be a boolean string, choices: "True", "False". Set to "True" by default.
* verify_ssl: must be a boolean string, choices: "True", "False". Set to "True" by default.

Data Using an AWS S3 URI
========================

Data class for loading datasets from AWS S3 URI. Specified by passing in
any valid bucket path that points to one of the valid data types.

`Data('s3a://my-bucket/file_name.txt')`

Possible `options`:

* `storage_options`: must be a dictionary where the keys for boto3 initialization are set
If `storage_options` is provided in `options`, the below variables are retrieved from the dictionary provided. Otherwise, will retrieve from `environment variables <https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#environment-variables>`_.

* `AWS_ACCESS_KEY_ID`
* `AWS_SECRET_ACCESS_KEY`
* `AWS_SESSION_TOKEN`
* `AWS_REGION` (default `us-east-1`)

0 comments on commit ec7fc9b

Please sign in to comment.