Skip to content

Commit

Permalink
Add the example DAG for azure blob storage (#1598)
Browse files Browse the repository at this point in the history
# Description
## What is the current behavior?
<!-- Please describe the current behavior that you are modifying. -->
- Missing documentation and example DAG for Azure blob storage

<!--
Issues are required for both bug fixes and features.
Reference it using one of the following:

closes: #ISSUE
related: #ISSUE
-->
closes: #1571

## What is the new behavior?
<!-- Please describe the behavior or changes that are being added by
this PR. -->

- Add the documentation and example DAG for Azure blob storage
(non-native path)

## Does this introduce a breaking change?
No

### Checklist
- [x] Created tests which fail without the change (if possible)
- [x] Extended the README / documentation, if necessary

(cherry picked from commit bb9a636)
  • Loading branch information
sunank200 authored and kaxil committed Jan 20, 2023
1 parent 5c0cc2f commit 9943e20
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 0 deletions.
9 changes: 9 additions & 0 deletions python-sdk/docs/astro/sql/operators/load_file.rst
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,15 @@ Users can also load data from an FTP:
:start-after: [START load_file_example_21]
:end-before: [END load_file_example_21]

Loading data from Azure Blob storage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Users can also load data from an Azure Blob storage:

.. literalinclude:: ../../../../example_dags/example_load_file.py
:language: python
:start-after: [START load_file_example_25]
:end-before: [END load_file_example_25]


Default Datasets
~~~~~~~~~~~~~~~~
Expand Down
13 changes: 13 additions & 0 deletions python-sdk/example_dags/example_load_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -330,4 +330,17 @@
)
# [END load_file_example_24]

# [START load_file_example_25]
aql.load_file(
input_file=File("wasb://astro-sdk/sample.csv", conn_id="wasb_default_conn"),
output_table=Table(
conn_id=SNOWFLAKE_CONN_ID,
metadata=Metadata(
database=os.environ["SNOWFLAKE_DATABASE"],
schema=os.environ["SNOWFLAKE_SCHEMA"],
),
),
)
# [END load_file_example_25]

aql.cleanup()

0 comments on commit 9943e20

Please sign in to comment.