Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-3153] [implementation] limit the size of fixtures for unit testing #8708

Open
1 task done
Tracked by #8283
graciegoheen opened this issue Sep 25, 2023 · 1 comment
Open
1 task done
Tracked by #8283
Labels
unit tests Issues related to built-in dbt unit testing functionality user docs [docs.getdbt.com] Needs better documentation

Comments

@graciegoheen
Copy link
Contributor

graciegoheen commented Sep 25, 2023

Housekeeping

  • I am a maintainer of dbt-core

Short description

From #8652 and #8290

We should add a limit to how large the unit testing fixtures can be, to avoid performance issues and ensure folks are using unit tests properly.

We should have a way to handle when folks are using seeds as their fixtures, and their seeds are larger than the fixture limit - if your seed file is larger than the size limit for fixture, you must define a fixture for your seed.

Acceptance criteria

  • error message if fixture is greater in size than 500 rows
  • error message if you use a seed as a fixture, and the seeds is greater in size than 500 rows
  • any number of columns is allowed
  • add tests that all data warehouses can support this max, if not, set the max lower for the given warehouse

note: it's ok if folks run into a warehouse constraint (like query length) for now, we will address that issue in a future ticket if necessary

Impact to Other Teams

None

Will backports be required?

No

Context

No response

@graciegoheen graciegoheen added the user docs [docs.getdbt.com] Needs better documentation label Sep 25, 2023
@github-actions github-actions bot changed the title [implementation] limit the size of fixtures for unit testing [CT-3153] [implementation] limit the size of fixtures for unit testing Sep 25, 2023
@graciegoheen
Copy link
Contributor Author

graciegoheen commented Oct 26, 2023

Notes from product/dx refinement:

@dbeatty10 dbeatty10 added the unit tests Issues related to built-in dbt unit testing functionality label Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
unit tests Issues related to built-in dbt unit testing functionality user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
Development

No branches or pull requests

3 participants