reach + frequency metrics

.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md

@@ -4,48 +4,22 @@
 **This PR will result in the following new package version:**
 <!--- Please add details around your decision for breaking vs non-breaking version upgrade. If this is a breaking change, were backwards-compatible options explored? -->
 
-**Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:**
+**Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:**
+<!--- Copy/paste the CHANGELOG for this version below. -->
 
 ## PR Checklist
 ### Basic Validation
 Please acknowledge that you have successfully performed the following commands locally:
-- [ ] dbt compile
-- [ ] dbt run –full-refresh
-- [ ] dbt run
-- [ ] dbt test
-- [ ] dbt run –vars (if applicable)
+- [ ] dbt run –full-refresh && dbt test
+- [ ] dbt run (if incremental models are present)
 
 Before marking this PR as "ready for review" the following have been applied:
-- [ ] The appropriate issue has been linked and tagged
-- [ ] You are assigned to the corresponding issue and this PR
+- [ ] The appropriate issue has been linked, tagged, and properly assigned
+- [ ] All necessary documentation and version upgrades have been applied
+- [ ] docs were regenerated (unless this PR does not include any code or yml updates)
 - [ ] BuildKite integration tests are passing
+- [ ] Detailed validation steps have been provided below
 
 ### Detailed Validation
-Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":
-- [ ] You have validated these changes and assure this PR will address the respective Issue/Feature.
-- [ ] You are reasonably confident these changes will not impact any other components of this package or any dependent packages.
-- [ ] You have provided details below around the validation steps performed to gain confidence in these changes.
-<!--- Provide the steps you took to validate your changes below. -->
-
-### Standard Updates
-Please acknowledge that your PR contains the following standard updates:
-- Package versioning has been appropriately indexed in the following locations:
-    - [ ] indexed within dbt_project.yml
-    - [ ] indexed within integration_tests/dbt_project.yml
-- [ ] CHANGELOG has individual entries for each respective change in this PR
-    <!--- If there is a parallel upstream change, remember to reference the corresponding CHANGELOG as an individual entry.  -->
-- [ ] README updates have been applied (if applicable)
-    <!--- Remember to check the following README locations for common updates. →
-        <!--- Suggested install range (needed for breaking changes) →
-        <!--- Dependency matrix is appropriately updated (if applicable) →
-        <!--- New variable documentation (if applicable) -->
-- [ ] DECISIONLOG updates have been updated (if applicable)
-- [ ] Appropriate yml documentation has been added (if applicable)
-
-### dbt Docs
-Please acknowledge that after the above were all completed the below were applied to your branch:
-- [ ] docs were regenerated (unless this PR does not include any code or yml updates)
-
-### If you had to summarize this PR in an emoji, which would it be?
-<!--- For a complete list of markdown compatible emojis check our this git repo (https://gist.github.com/rxaviers/7360908)  --> 
-:dancer:
+Please share any and all of your validation steps:
+<!--- Provide the steps you took to validate your changes below. -->

CHANGELOG.md

@@ -1,3 +1,13 @@
+# dbt_facebook_ads_source v0.7.3
+[PR #35](https://github.com/fivetran/dbt_facebook_ads_source/pull/35) includes the following updates:
+
+## Bug Fixes
+- Resolved the "duplicate column" error that would arise when the `facebook_ads__basic_ad_passthrough_metrics` variable included `reach` or `frequency`. `Reach` and `frequency` were included by default in the `stg_facebook_ads__basic_ad` model but will now only persist to downstream transform models if specified in the `facebook_ads__basic_ad_passthrough_metrics` variable.
+  - This error is now avoided by first [checking](https://github.com/fivetran/dbt_facebook_ads_source/blob/main/models/stg_facebook_ads__basic_ad.sql#L41-L45) if `reach` and `frequency` are included in `facebook_ads__basic_ad_passthrough_metrics` and only selecting the columns once if so.
+
+## Under the Hood
+- Updated the maintainer PR template to our most up-to-date standards.
+
 # dbt_facebook_ads_source v0.7.2
 [PR #33](https://github.com/fivetran/dbt_facebook_ads_source/pull/33) includes the following updates:
 
@@ -20,13 +30,9 @@
 - Updated tmp models to union source data using the `fivetran_utils.union_data` macro. 
 - To distinguish which source each field comes from, added `source_relation` column in each staging model and applied the `fivetran_utils.source_relation` macro.
 - Updated tests to account for the new `source_relation` column.
-
-# dbt_facebook_ads_source v0.UPDATE.UPDATE
-
- ## Under the Hood:
-
 - Incorporated the new `fivetran_utils.drop_schemas_automation` macro into the end of each Buildkite integration test job.
 - Updated the pull request [templates](/.github).
+
 # dbt_facebook_ads_source v0.6.0
 
 ## 🚨 Breaking Changes 🚨:

dbt_project.yml

@@ -1,5 +1,5 @@
 name: 'facebook_ads_source'
-version: '0.7.2'
+version: '0.7.3'
 config-version: 2
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 models:

integration_tests/dbt_project.yml

@@ -1,9 +1,14 @@
 name: 'facebook_ads_source_integration_tests_2'
-version: '0.7.2'
+version: '0.7.3'
 profile: 'integration_tests'
 config-version: 2
 
 vars:
+  facebook_ads__basic_ad_passthrough_metrics:
+    - name: "reach"
+    - name: "frequency"
+      alias: "fake_freq_name"
+
   facebook_ads_schema: facebook_ads_source_integration_tests_2
   facebook_ads_source:
     facebook_ads_account_history_identifier: "facebook_ads_account_history_data"

macros/get_basic_ad_columns.sql

@@ -13,7 +13,19 @@
     {"name": "frequency", "datatype": dbt.type_float()}
 ] %}
 
-{{ fivetran_utils.add_pass_through_columns(columns, var('facebook_ads__basic_ad_passthrough_metrics')) }}
+{# 
+    Reach and Frequency are not included in downstream models by default, though they are included in the staging model.
+    The below ensures that users can add Reach and Frequency to downstream models with the `facebook_ads__basic_ad_passthrough_metrics` variable
+    while avoiding duplicate column errors.
+ #}
+{% set unique_passthrough = [] %}
+{% for field in var('facebook_ads__basic_ad_passthrough_metrics') %}
+    {% if (field.alias if field.alias else field.name)|lower not in ('reach', 'frequency') %}
+        {% do unique_passthrough.append({"name": field.name, "alias": field.alias}) %}
+    {% endif %}
+{% endfor %}
+
+{{ fivetran_utils.add_pass_through_columns(columns, unique_passthrough) }}
 
 {{ return(columns) }}
 

models/stg_facebook_ads__basic_ad.sql

@@ -36,9 +36,26 @@ final as (
         cast(account_id as {{ dbt.type_bigint() }}) as account_id,
         impressions,
         coalesce(inline_link_clicks,0) as clicks,
-        spend,
-        reach,
-        frequency
+        spend
+
+        {# 
+            Reach and Frequency are not included in downstream models by default, though they are included in the staging model.
+            The below ensures that users can add Reach and Frequency to downstream models with the `facebook_ads__basic_ad_passthrough_metrics` variable
+            while avoiding duplicate column errors.
+        #}
+        {%- set check = [] %}
+        {%- for field in var('facebook_ads__basic_ad_passthrough_metrics') -%}
+            {%- set field_name = field.alias|default(field.name)|lower %}
+            {% if field_name in ['reach', 'frequency'] %}
+                {% do check.append(field_name) %}
+            {% endif %}
+        {%- endfor %}
+
+        {%- for metric in ['reach', 'frequency'] -%}
+            {% if metric not in check %}
+                , {{ metric }}
+            {% endif %}
+        {%- endfor %}
 
         {{ fivetran_utils.fill_pass_through_columns('facebook_ads__basic_ad_passthrough_metrics') }}
     from fields