This project is a scalable data pipeline built with Python and SQL. It transforms, analyzes, and exports data from the Pima Indians Diabetes Dataset. The pipeline follows best practices for scalability and efficiency, simulating real-world challenges in Data Engineering. It includes steps like data ingestion, SQL transformations, and exporting results to CSV files.
- Scalable: Handles large datasets and adapts to growing data.
- ETL Process: Extract, transform, and load data in a structured way.
- Integration: Combines Python and SQL for smooth data processing.
- Export Results: Saves processed data in a CSV file for analysis.