This project aims to retrieve the metadata generated by dlt during pipeline runs executed by Apache Airflow and surface it via Airflow's OpenLineage integration. This metadata should be viewable through Marquez.
This project structure has been adapted from the Astronomer tutorial for ease of running Marquez locally: Integrate OpenLineage and Airflow with Marquez
The dlt pipeline has been taken from their Load Data from a SQL Database tutorial, which loads data from the public MySQL RFam database into a local DuckDB instance.