Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug/add-disable-teams #32

Merged
merged 5 commits into from
Feb 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,19 @@
steps:
- label: ":postgres: Run Tests - Postgres"
key: "run-dbt-postgres"
plugins:
- docker#v3.13.0:
image: "python:3.8"
shell: [ "/bin/bash", "-e", "-c" ]
environment:
- "BASH_ENV=/tmp/.bashrc"
- "CI_POSTGRES_DBT_DBNAME"
- "CI_POSTGRES_DBT_HOST"
- "CI_POSTGRES_DBT_PASS"
- "CI_POSTGRES_DBT_USER"
commands: |
bash .buildkite/scripts/run_models.sh postgres

- label: ":snowflake-db: Run Tests - Snowflake"
key: "run_dbt_snowflake"
plugins:
Expand Down Expand Up @@ -43,7 +58,7 @@ steps:
commands: |
bash .buildkite/scripts/run_models.sh redshift

- label: ":bricks: Run Tests - Databricks"
- label: ":databricks: Run Tests - Databricks"
key: "run_dbt_databricks"
plugins:
- docker#v3.13.0:
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
# dbt_github_source v0.7.0
## 🚨 Breaking Change 🚨
- Expanded the variable `github__using_repo_team` to also toggle on/off the `TEAM` table along with and in addition to the `REPO_TEAM` table. This was done to fix issues with dependent models in our transform package, where `TEAM` is only ever used in conjunction with `REPO_TEAM`. ([#32](https://github.com/fivetran/dbt_github_source/pull/32))

## 🎉 Features 🎉
- PostgreSQL compatibility! ([#32](https://github.com/fivetran/dbt_github_source/pull/32))
# dbt_github_source v0.6.0
[PR #26](https://github.com/fivetran/dbt_github_source/pull/26) includes the following breaking changes:
## 🚨 Breaking Changes 🚨:
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
## Step 1: Prerequisites
To use this dbt package, you must have the following:
- At least one Fivetran Github connector syncing data into your destination.
- A **BigQuery**, **Snowflake**, **Redshift** or **Databricks** destination.
- A **BigQuery**, **Snowflake**, **Redshift**, **PostgreSQL** or **Databricks** destination.

### Databricks Dispatch Configuration
If you are using a Databricks destination with this package you will need to add the below (or a variation of the below) dispatch configuration within your `dbt_project.yml`. This is required in order for the package to accurately search for macros within the `dbt-labs/spark_utils` then the `dbt-labs/dbt_utils` packages respectively.
Expand All @@ -40,7 +40,7 @@ Include the following github_source package version in your `packages.yml` file.
```yaml
packages:
- package: fivetran/github_source
version: [">=0.6.0", "<0.7.0"]
version: [">=0.7.0", "<0.8.0"]
```

## Step 3: Define database and schema variables
Expand All @@ -55,7 +55,7 @@ vars:
## Step 4: Disable models for non-existent sources
Your Github connector might not sync every table that this package expects. If your syncs exclude certain tables, it is because you either don't use that functionality in Github or have actively excluded some tables from your syncs.

If you do not have the `REPO_TEAM` table synced, add the following variable to your `dbt_project.yml` file:
If you do not have the `TEAM` and `REPO_TEAM` tables synced, add the following variable to your `dbt_project.yml` file:

```yml
vars:
Expand Down
2 changes: 1 addition & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
config-version: 2
name: 'github_source'
version: '0.6.0'
version: '0.7.0'
require-dbt-version: [">=1.3.0", "<2.0.0"]
models:
github_source:
Expand Down
2 changes: 1 addition & 1 deletion docs/catalog.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/index.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/manifest.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/run_results.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'github_source_integration_tests'
version: '0.6.0'
version: '0.7.0'
config-version: 2
profile: 'integration_tests'
vars:
Expand Down
1 change: 1 addition & 0 deletions models/stg_github__team.sql
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{{ config(enabled=var('github__using_repo_team', True)) }}

with base as (

Expand Down
2 changes: 2 additions & 0 deletions models/tmp/stg_github__team_tmp.sql
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
{{ config(enabled=var('github__using_repo_team', True)) }}

select *
from {{ var('team') }}