An ETL Data Pipelines Project that uses AirFlow DAGs to extract accessories and jewelry data from PostgreSQL Schemas and the shoes data from a CSV file, load them in AWS Data Lake, transform them with Python script, and finally load them into SnowFlake Data warehouse using SCD type 2.
python
aws
postgres
airflow
etl
s3-bucket
pandas
data-warehouse
slowly-changing-dimensions
snowfalek
-
Updated
Nov 25, 2023 - Python