Creating pipelines using Python3 and Apache Airflow to load tables into Google Big Query Dataware House
-
Updated
Nov 6, 2021 - Python
Google BigQuery enables companies to handle large amounts of data without having to manage infrastructure. Google’s documentation describes it as a « serverless architecture (that) lets you use SQL queries to answer your organization's biggest questions with zero infrastructure management. BigQuery's scalable, distributed analysis engine lets you query terabytes in seconds and petabytes in minutes. » Its client libraries allow the use of widely known languages such as Python, Java, JavaScript, and Go. Federated queries are also supported, making it flexible to read data from external sources.
📖 A highly rated canonical book on it is « Google BigQuery: The Definitive Guide », a comprehensive reference.
Another enriching read on the subject is the inside story told in the article by the founding product manager of BigQuery celebrating its 10th anniversary.
Creating pipelines using Python3 and Apache Airflow to load tables into Google Big Query Dataware House
Automate Google Forms Reponses on Google Sheets to be Imported in Bigquery & Automate Building Parameterized Views.
Released May 19, 2010