Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1591] Don't parse empty Python files #6433

Merged
merged 4 commits into from
Dec 20, 2022
Merged

[CT-1591] Don't parse empty Python files #6433

merged 4 commits into from
Dec 20, 2022

Conversation

aranke
Copy link
Member

@aranke aranke commented Dec 13, 2022

Resolves #6367

Description

Only parse Python AST if body is not empty

Checklist

@aranke aranke requested review from a team as code owners December 13, 2022 09:23
@cla-bot cla-bot bot added the cla:yes label Dec 13, 2022
Base automatically changed from ct-1284 to main December 13, 2022 23:07
@aranke
Copy link
Member Author

aranke commented Dec 20, 2022

I've confirmed the following, I think PR should be good to merge?

  1. This PR resolves the issue (see 1 and 2).
  2. The empty Python model is ignored during dbt run2 and dbt ls3, just like an empty SQL model.
  3. The empty Python model4 is shown in the dbt DAG, just like an empty SQL model.

P.S. I misclicked on closing this PR, hence the noise below.

Footnotes

  1. Old dbt run

    ❯ dbt run
    06:14:26  Running with dbt=1.3.1
    06:14:27  Encountered an error:
    Parsing Error in model empty (models/challenges/py/empty.py)
      dbt only allow one model defined per python file
    
  2. New dbt run:

    ❯ dbt run
    06:14:49  Running with dbt=1.4.0-b1
    06:14:49  Unable to do partial parsing because a project config has changed
    06:14:50  Found 6 models, 4 tests, 0 snapshots, 0 analyses, 415 macros, 0 operations, 0 seed files, 6 sources, 0 exposures, 0 metrics
    06:14:50  Using default selector default
    06:14:50
    06:14:51  Concurrency: 8 threads (target='snowflake')
    06:14:51
    06:14:51  1 of 5 START sql table model dbt_karanke.stg_customers ......................... [RUN]
    06:14:51  2 of 5 START sql table model dbt_karanke.stg_orders ............................ [RUN]
    06:14:55  1 of 5 OK created sql table model dbt_karanke.stg_customers .................... [SUCCESS 1 in 3.02s]
    06:14:55  3 of 5 START python table model dbt_karanke.int_customers ...................... [RUN]
    06:15:07  2 of 5 OK created sql table model dbt_karanke.stg_orders ....................... [SUCCESS 1 in 15.75s]
    06:15:10  3 of 5 OK created python table model dbt_karanke.int_customers ................. [SUCCESS 1 in 15.42s]
    06:15:10  4 of 5 START python table model dbt_karanke.int_orders ......................... [RUN]
    06:16:08  4 of 5 OK created python table model dbt_karanke.int_orders .................... [SUCCESS 1 in 58.26s]
    06:16:08  5 of 5 START python table model dbt_karanke.unfuzz_py .......................... [RUN]
    06:16:40  5 of 5 OK created python table model dbt_karanke.unfuzz_py ..................... [SUCCESS 1 in 32.04s]
    06:16:40
    06:16:40  Finished running 5 table models in 0 hours 1 minutes and 50.46 seconds (110.46s).
    06:16:40
    06:16:40  Completed successfully
    06:16:40
    06:16:40  Done. PASS=5 WARN=0 ERROR=0 SKIP=0 TOTAL=5
    
    2
  3. New dbt ls:

    ❯ dbt ls
    demo_data.intermediate.int_customers
    demo_data.intermediate.int_orders
    demo_data.staging.stg_customers
    demo_data.staging.stg_orders
    demo_data.challenges.py.unfuzz_py
    demo_data.staging.not_null_stg_customers_customer_id
    demo_data.staging.not_null_stg_orders_order_id
    demo_data.staging.unique_stg_customers_customer_id
    demo_data.staging.unique_stg_orders_order_id
    
  4. New dbt DAG:
    dbt-dag

@aranke aranke closed this Dec 20, 2022
@aranke aranke reopened this Dec 20, 2022
@aranke aranke requested a review from ChenyuLInx December 20, 2022 06:41
Copy link
Contributor

@ChenyuLInx ChenyuLInx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!! Thanks for testing out the behavior of run, list and doc

@aranke aranke merged commit ce9d0af into main Dec 20, 2022
@aranke aranke deleted the ct-1591 branch December 20, 2022 18:56
@hope-wat
Copy link

With this change now being live should we not be seeing this behavior anymore in dbt cloud?

I just created a new .py file and this threw the same problem. I was the original poster/opened the bug with the corresponding loom so I am not resposting here in the comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-1591] [Bug] New python files .py cause parsing and compilation errors
3 participants