Mini MDS

Lightweight, open source, locally-hosted Modern Data Stack

• Extract & Load: dlt
• Data Quality: Great Expectations
• Storage: DuckDB
• Transformation: dbt
• Orchestration: Prefect
• Visualization: Dash

Installation

Prerequisites: Install git and uv.

Clone repository and change directory:

git clone https://github.com/esadek/mini-mds.git
cd mini-mds

Create and activate virtual environment:

uv venv
source .venv/bin/activate

Install required packages:

uv sync

Add dbt connection profile:

uv run scripts/add_profile.py

Usage

Extract, validate, load and transform data:

uv run prefect/elt.py

Visualize data:

uv run dash/app.py

Project Structure

mini-mds
├── .github/                # GitHub workflows
├── dash/                   # Dash application
├── dbt/                    # dbt project
├── duckdb/                 # DuckDB warehouse
├── prefect/                # Prefect workflows
├── scripts/                # Scripts
├── .gitignore              # Untracked files to ignore
├── .python-version         # Default Python version
├── LICENSE                 # MIT license
├── pyproject.toml          # Project metadata
├── README.md               # Documentation
└── uv.lock                 # Dependency lockfile