Skip to content
This repository was archived by the owner on Dec 19, 2023. It is now read-only.

Commit 26dcb63

Browse files
afranziafranzi
authoredJan 4, 2022
feat: [DAT-3898] - Prepare 💀 project structure (#1)
* feat: [DAT-3898] - Prepare 💀 project structure * fix: [DAT-3898] - 📝 Use the proper license notation - Apache-2.0 * feat: [DAT-3898] - 📝 Add PR template * fix: [DAT-3898] - 🔧 Use the latest ci-standard-checks version * doc: [DAT-3898] - 📝 Add a 1st Readme version * fix: [DAT-3898] - 🔧 Fix typo on github action badges
1 parent bc3ac40 commit 26dcb63

15 files changed

+2695
-0
lines changed
 

‎.flake8

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[flake8]
2+
max-line-length = 120
3+
extend-ignore = F821,F841,E203

‎.github/CODEOWNERS

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Add by default all the Data Engineering Team
2+
* @Typeform/data-engineering

‎.github/PULL_REQUEST_TEMPLATE.md

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
## Purpose
2+
_Describe the problem or feature in addition to a link to the issues._

‎.github/workflows/ci-deploy.yml

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Deploy to Pypi
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
8+
jobs:
9+
deploy:
10+
runs-on: ubuntu-latest
11+
12+
steps:
13+
- uses: actions/checkout@v2.3.5
14+
15+
- name: Setup Python
16+
uses: actions/setup-python@v2.2.2
17+
with:
18+
python-version: "3.7"
19+
20+
- name: Install Poetry & Tox
21+
run: pip install poetry tox
22+
23+
- name: Install Dependencies
24+
run: poetry install
25+
26+
- name: Run Tox
27+
run: tox -e py37
28+
29+
- name: Poetry Publish
30+
run: |
31+
poetry build -f sdist
32+
poetry publish
33+
env:
34+
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_TOKEN }}

‎.github/workflows/ci-standard-checks.yml

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
name: CI Standard Checks
2+
on:
3+
push:
4+
branches:
5+
- main
6+
pull_request:
7+
types: [opened, edited, synchronize, reopened]
8+
branches:
9+
- main
10+
11+
jobs:
12+
ci-standard-checks:
13+
runs-on: ubuntu-latest
14+
15+
steps:
16+
- name: Check Out Source Code
17+
uses: actions/checkout@v2
18+
with:
19+
fetch-depth: 0
20+
21+
- name: CI Standard Checks
22+
uses: Typeform/ci-standard-checks@v1-beta
23+
with:
24+
githubToken: ${{ secrets.GITHUB_TOKEN }}
25+
dockerUsername: ${{ secrets.GITLEAKS_DOCKER_USERNAME }}
26+
dockerPassword: ${{ secrets.GITLEAKS_DOCKER_PASSWORD }}

‎.github/workflows/ci-test.yml

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: CI PyTests
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
pull_request:
8+
9+
jobs:
10+
tests:
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- uses: actions/checkout@v2.3.5
15+
16+
- name: Setup Python
17+
uses: actions/setup-python@v2.2.2
18+
with:
19+
python-version: "3.7"
20+
21+
- name: Install Poetry & Tox
22+
run: pip install poetry tox
23+
24+
- name: Install Dependencies
25+
run: poetry install
26+
27+
- name: Run Tox
28+
run: tox -e py37

‎.gitignore

+103
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
*.DS_Store
6+
7+
# C extensions
8+
*.so
9+
10+
# Distribution / packaging
11+
.Python
12+
env/
13+
build/
14+
develop-eggs/
15+
dist/
16+
downloads/
17+
eggs/
18+
.eggs/
19+
lib/
20+
lib64/
21+
parts/
22+
sdist/
23+
var/
24+
wheels/
25+
*.egg-info/
26+
.installed.cfg
27+
*.egg
28+
.idea
29+
30+
# PyInstaller
31+
# Usually these files are written by a python script from a template
32+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
33+
*.manifest
34+
*.spec
35+
36+
# Installer logs
37+
pip-log.txt
38+
pip-delete-this-directory.txt
39+
40+
# Unit test / coverage reports
41+
htmlcov/
42+
.tox/
43+
.coverage
44+
.coverage.*
45+
.cache
46+
nosetests.xml
47+
coverage.xml
48+
*.cover
49+
.hypothesis/
50+
51+
# Translations
52+
*.mo
53+
*.pot
54+
55+
# Django stuff:
56+
*.log
57+
local_settings.py
58+
59+
# Flask stuff:
60+
instance/
61+
.webassets-cache
62+
63+
# Scrapy stuff:
64+
.scrapy
65+
66+
# Sphinx documentation
67+
docs/_build/
68+
69+
# PyBuilder
70+
target/
71+
72+
# Jupyter Notebook
73+
.ipynb_checkpoints
74+
75+
# pyenv
76+
.python-version
77+
78+
# celery beat schedule file
79+
celerybeat-schedule
80+
81+
# SageMath parsed files
82+
*.sage.py
83+
84+
# dotenv
85+
.env
86+
87+
# virtualenv
88+
.venv
89+
venv/
90+
ENV/
91+
92+
# Spyder project settings
93+
.spyderproject
94+
.spyproject
95+
96+
# Rope project settings
97+
.ropeproject
98+
99+
# mkdocs documentation
100+
/site
101+
102+
# mypy
103+
.mypy_cache/

‎README.md

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# DAG Bakery - WIP :wrench:
2+
3+
![deploy](https://github.com/typeform/dag-bakery/actions/workflows/ci-deploy.yml/badge.svg?branch=main)
4+
![standard-checks](https://github.com/typeform/dag-bakery/actions/workflows/ci-standard-checks.yml/badge.svg?branch=main)
5+
![tests](https://github.com/typeform/dag-bakery/actions/workflows/ci-test.yml/badge.svg?branch=main)
6+
7+
> `dag-bakery` aims to simplify our DAG development by removing all the boilerplate and duplicated code when defining
8+
> multiple DAG cross teams. All we need is modularized tasks and yaml code.
9+
10+
## Roadmap
11+
12+
- [x] : Prepare skeleton & workflows.
13+
- [ ] : Autogenerate documentation.
14+
- [ ] : Develop the DAG generator with Airflow 1.10.15.
15+
- [ ] : Design a project logo.
16+
- [ ] : Support Airflow 2.x.x

‎dag_bakery/__init__.py

Whitespace-only changes.

‎dag_bakery/dummy.py

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
def potato():
2+
return "tomato"

‎poetry.lock

+2,435
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎pyproject.toml

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
[tool.poetry]
2+
name = "dag-bakery"
3+
version = "0.0.1"
4+
description = "Dag Bakery enables the capability to define Airflow DAGs via YAML"
5+
authors = ["Data Platform <data.platform@typeform.com>"]
6+
license = "Apache-2.0"
7+
8+
[tool.poetry.dependencies]
9+
python = "^3.7"
10+
apache-airflow = "1.10.15"
11+
12+
[tool.poetry.dev-dependencies]
13+
black = "^21.12b0"
14+
pytest = "^6.2.5"
15+
pytest-cov = "^3.0.0"
16+
flake8 = "^4.0.1"
17+
tox = "^3.24.5"
18+
19+
[tool.black]
20+
line-length = 120
21+
22+
[build-system]
23+
requires = ["poetry-core>=1.0.0"]
24+
build-backend = "poetry.core.masonry.api"

‎tests/__init__.py

Whitespace-only changes.

‎tests/test_dummy.py

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
import unittest
2+
from dag_bakery.dummy import potato
3+
4+
5+
class TestDummy(unittest.TestCase):
6+
def test_potato_tomato(self):
7+
self.assertEqual(potato(), "tomato")

‎tox.ini

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
[tox]
2+
skipsdist = true
3+
envlist = py37
4+
5+
[testenv]
6+
setenv = PYTHONPATH = {toxinidir}/src
7+
whitelist_externals = poetry
8+
commands =
9+
poetry install -v
10+
poetry run pytest -v --cov
11+
flake8 dag_bakery/ tests/
12+
black --check .
13+

0 commit comments

Comments
 (0)
This repository has been archived.