Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1933] [Bug] Time ingestion partitioning doesn't work with date data_type #486

Closed
2 tasks done
github-christophe-oudar opened this issue Jan 27, 2023 · 2 comments · Fixed by #496
Closed
2 tasks done
Labels
type:bug Something isn't working

Comments

@github-christophe-oudar
Copy link
Contributor

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When using "time_ingestion_partitioning": true and "data_type": "date", a dbt run will fail with following error

BadRequest('PARTITION BY expression must be _PARTITIONDATE, DATE(_PARTITIONTIME), DATE(<timestamp_column>), DATE(<datetime_column>), DATETIME_TRUNC(<datetime_column>, DAY/HOUR/MONTH/YEAR), a DATE column, TIMESTAMP_TRUNC(<timestamp_column>, DAY/HOUR/MONTH/YEAR), DATE_TRUNC(<date_column>, MONTH/YEAR), or RANGE_BUCKET(<int64_column>, GENERATE_ARRAY(<int64_value>, <int64_value>[, <int64_value>]))')

Expected Behavior

Expected behavior is that the model is created and working as intended.

Steps To Reproduce

Using following model

{{
    config(
        materialized = 'incremental',
        incremental_strategy='insert_overwrite',
        partition_by = {
          "field": "day",
          "data_type": "date",
          "granularity": "day",
          "time_ingestion_partitioning": true
        }
    )
}}
SELECT DATE('2023-01-25') as day, 1 as value

The failure can be reproduced

Relevant log output

No response

Environment

- OS: MacOS 12.6
- Python: 3.10.6
- dbt-core: 1.4.0-rc1
- dbt-bigquery: 1.4.0-rc1

Additional Context

No response

@github-actions github-actions bot changed the title [Bug] Time ingestion partitioning doesn't work with date data_type [CT-1933] [Bug] Time ingestion partitioning doesn't work with date data_type Jan 27, 2023
@dbeatty10
Copy link
Contributor

Thanks for opening a PR for this as well @github-christophe-oudar !

@VersusFacit
Copy link
Contributor

VersusFacit commented Apr 26, 2023

This issue will not be the end of this saga. This library has lots of edge cases and mutually exclusive DDL elements (no AS + PARTITION BY). To get total coverage, a new issue will need opening with the next most relevant use case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants