Dataproc Scala Examples is an effort to assist in the creation of Spark jobs written in Scala to run on Dataproc.

Spark & Airflow demo for meetup

Coronavirus Dashboard

Проект № 2 для курса "Инженер данных".

An End to End solution to read XML data from FTP server and process and import them into postgres relational database

Implementation of data processing in the MapReduce model.

This repository contains the spark ML code which will consume the customer's credit card transactions data from the database through spark streaming and the ML model will predict whether the credit transaction is fraud or not which will be shown on the dashboard.

🚦 Project of Data Warehouse in star architecture based on UK traffic data

Analysis of tweets related to Covid-19 extracted from Twitter API configured to run on Airflow

Credit card fraud detection scans prior spending habits and showcases how much amount is spend, at which merchant, at what frequency, etc. Based on this learning, current credit transaction will be predicted as fraud if it deviates from previously noted transaction.s.

A comprehensive ELT (Extract, Load, Transformation) project using Apache Spark, Apache Hive and DBT

Spatial parking optimized tracking system to avoid parking ticket

Batch ETL data pipeline built on HDP 3.0 to process daily sales and business data to procedure power Bi reports. Automated the pipelines using Airflow.