The ETL pipeline for Stock info.

Support code for the article "Connecting GCP Dataproc and Elasticsearch: Bridging the Worlds of Big Data and (vector) Search"

Orchestrating ETL pipeline in Google cloud using Apache Airflow

Under construction....

Data Workflows with GCP Dataproc, Apache Airflow and Apache Spark

Repositório para armazenar artefatos de um trabalho da disciplina de Computação Distribuída.

Generando un proceso ETL con dataset de Amazon

Big data analysis of 'shared-world' cloud application.

A basic implementation of hadoop cluster using GCloud DataProc

Inventory value is also important for determining a company's liquidity, or its ability to meet its short-term financial obligations. A high inventory value can indicate that a company has too much money tied up in inventory, which could make it difficult for the company to pay its bills.

数据处理工作流，elt

Google DataProc Spark Scala Job for MNIST Handwritten Digit Recognition using Decision Trees (Spark MLlib)

Orchestration Dataproc serverless job with Airflow

DataTalksClub Data Engineering Zoomcamp Project

Creating a robust and scalable data pipeline on Google Cloud Platform (GCP) to monitor and analyze stock performance. Leveraging the power of GCP's data processing and storage services, a comprehensive solution has been built to efficiently collect, process, and visualize stock data.

Collected data about from three sources, one opinion-based social media in twitter, research data in New York Times, and the third is the common crawl data for the same topic or key phrase, and from similar time periods. Processed the three data sets collected individually using classical big data methods like Map Reduce in Google Dataproc Clust…

This repository is created automation for processing local data to Cloud BigQuery using Cloud Dataproc

A Pyspark project that performs ETL on a Dataproc cluster and writes data to Google Cloud Storage/BigQuery.