This repository includes some AWS Cloud Quest. it not include the cloud practitioner labs

Build a Visualization and Monitoring Dashboard for IoT Data with Amazon Kinesis Analytics and Amazon QuickSight

you run a script to mimic multiple sensors publishing messages on an IoT MQTT topic, with one message published every second. The events get sent to AWS IoT, where an IoT rule is configured. The IoT rule captures all messages and sends them to Firehose. From there, Firehose writes the messages in batches to objects stored in S3. In S3, you set u…

A demand forecasting pipeline deployed on Azure and AWS

This project repo 📺 offers a robust solution meticulously crafted to efficiently manage, process, and analyze YouTube video data leveraging the power of AWS services. Whether you're diving into structured statistics or exploring the nuances of trending key metrics, this pipeline is engineered to handle it all with finesse.

Unveiling job market trends with Scrapy and AWS

Smart City Realtime Data Engineering Project

The testbed showing how to embed QuickSight dashboards into a web app

This project demonstrates a complete data pipeline for extracting, transforming, and loading (ETL) Reddit data into an Amazon Redshift data warehouse. The pipeline uses various AWS services and tools including Apache Airflow, PostgreSQL, AWS S3, AWS Glue, AWS Athena, and Amazon Redshift. The project is orchestrated using Docker and Apache Airflow

Refresh Amazon QuickSight SPICE Datasets

Streaming data pipeline on aws, Tech session repository for hist

Daily power consumption and production data retrieving and insights from France.

This project aims to securely manage, streamline, and perform analysis on the structured and semi-structured YouTube videos data based on the video categories and the trending metrics.

Dashboard generated using Python and AWS services

Ambiental telemetry (temperature and humidity) at the edge using AWS Greengrass v2 on Raspberry PI and ESP32 client devices

Data Engineering Final Project - June 23, 2022

Get the dataset intro a S3 bucket, use AWS glue to transform the dataset, write a Lambda script to clean the dataset, query the dataset via AWS Athena then build a dashboard using AWS Quicksight.

Batch datapipeline on aws using glue

An end-to-end data engineering project in which five NYC DOT datasets were modified in an ETL process and analyzed for insights.