Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement chained comparison improvements and related checks #7611

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

areveny
Copy link
Contributor

@areveny areveny commented Oct 12, 2022

Hi.

This PR adds improvements to chained-comparison by processing the chain of comparisons into a graph. It's still a bit unpolished but I would like some higher-level feedback on the direction.

It adds two new checks:

  1. impossible-comparison for cases when a comparison would always be false, e.g. a > b and b > a or a == 5 and a > b and b == 1.

  2. comparison-all-equal for a special cycle case where every comparison allows equality. a >= b >= c and c >= a simplifies to a == b == c.

It modifies chained-comparison to be more accurate, specifically the case a < b < c and a < c should be simplified to just a < b < c. It currently is not.

It also prints what simplified comparison is.

Unfortunately, this is at the cost of complexity. The algorithm finds the minimum number of comparisons by finding the longest path through the graph of comparisons repeatedly, recalculating all the path lengths after each path is emitted. (I could not get it working by emitting paths in arbitrary order until all nodes are accounted for because of the case above, a < b < c and a < c.) The overhead of doing so should be limited by the reality that most comparisons tend to have just a few statements.

Please let me know what you think. I'm open to big suggestions, but want to share my code for feedback and to keep this moving.

Type of Changes

Type
✨ New feature

Description

Closes #5814

@coveralls
Copy link

coveralls commented Oct 12, 2022

Pull Request Test Coverage Report for Build 3231501930

  • 143 of 143 (100.0%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.03%) to 95.357%

Totals Coverage Status
Change from base Build 3225859940: 0.03%
Covered Lines: 17231
Relevant Lines: 18070

💛 - Coveralls

@github-actions
Copy link
Contributor

🤖 Effect of this PR on checked open source code: 🤖

Effect on pandas:
The following messages are now emitted:

  1. chained-comparison:
    Simplify chained comparison between the operands: neg_ct == 0 > result
    https://github.com/pandas-dev/pandas/blob/cdb905a8b7e25433a3268951fc3ec68aa03914aa/pandas/core/_numba/kernels/mean_.py#L135

The following messages are no longer emitted:

  1. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/pandas-dev/pandas/blob/cdb905a8b7e25433a3268951fc3ec68aa03914aa/pandas/tests/io/formats/test_info.py#L119
  2. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/pandas-dev/pandas/blob/cdb905a8b7e25433a3268951fc3ec68aa03914aa/pandas/core/algorithms.py#L1361
  3. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/pandas-dev/pandas/blob/cdb905a8b7e25433a3268951fc3ec68aa03914aa/pandas/core/construction.py#L675
  4. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/pandas-dev/pandas/blob/cdb905a8b7e25433a3268951fc3ec68aa03914aa/pandas/core/arrays/_ranges.py#L124
  5. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/pandas-dev/pandas/blob/cdb905a8b7e25433a3268951fc3ec68aa03914aa/pandas/core/arrays/_ranges.py#L129
  6. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/pandas-dev/pandas/blob/cdb905a8b7e25433a3268951fc3ec68aa03914aa/pandas/plotting/_matplotlib/misc.py#L204
  7. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/pandas-dev/pandas/blob/cdb905a8b7e25433a3268951fc3ec68aa03914aa/pandas/plotting/_matplotlib/misc.py#L213

Effect on sentry:
The following messages are now emitted:

  1. chained-comparison:
    Simplify chained comparison between the operands: 300 > code >= 200
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/runner/formatting.py#L27
  2. chained-comparison:
    Simplify chained comparison between the operands: 500 > code >= 400
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/runner/formatting.py#L29
  3. chained-comparison:
    Simplify chained comparison between the operands: fcp_maximum_threshold > fcp >= fcp_minimum_threshold
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/utils/performance_issues/performance_detection.py#L737
  4. chained-comparison:
    Simplify chained comparison between the operands: metrics > 10 >= sessions
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/release_health/duplex.py#L254
  5. chained-comparison:
    Simplify chained comparison between the operands: sessions > 10 >= metrics
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/release_health/duplex.py#L254

The following messages are no longer emitted:

  1. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/runner/formatting.py#L27
  2. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/runner/formatting.py#L29
  3. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/api/endpoints/project_ownership.py#L52
  4. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/api/endpoints/codeowners/__init__.py#L46
  5. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/utils/performance_issues/performance_detection.py#L737
  6. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/release_health/duplex.py#L254
  7. chained-comparison:
    Simplify chained comparison between the operands
    https://github.com/getsentry/sentry/blob/717b4e740dcc7a6a40263ce3a1fbad1bf87ea592/src/sentry/release_health/duplex.py#L254

This comment was generated for commit f41f4cb

@Pierre-Sassoulas Pierre-Sassoulas added Enhancement ✨ Improvement to a component Needs review 🔍 Needs to be reviewed by one or multiple more persons labels Oct 12, 2022
Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the work @areveny. There's a lot to review but from what I can see after a surface review it's possible that making the gragh check's code independent from the RefactoringChecker might be a good idea, because 1) There's a lot of code and maintenance would benefit from it being separated 2) It's going to be easier if we want to extends usiing-constant-testsdirectly where it's at instead of refactoring how the checker work to be able to raise any message anywhere.

pylint/checkers/refactoring/refactoring_checker.py Outdated Show resolved Hide resolved
"Emitted when items in a boolean condition are all <= or >="
"This is equivalent to asking if they all equal.",
),
"R1738": (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should raise the existing message for constant comparison using-constant-test (https://pylint.pycqa.org/en/latest/user_guide/messages/warning/using-constant-test.html#using-constant-test-w0125). We might need to relax some constraint on what checker can do, I'm not sure a checker can raise messages it does not define right now.

@jacobtylerwalls
Copy link
Member

Hi @areveny, just wondering: are the false negatives in the primer result one of the items you were hoping to polish after getting an initial review?

@Pierre-Sassoulas Pierre-Sassoulas added the Needs take over 🛎️ Orignal implementer went away but the code could be salvaged. label Jan 16, 2023
pylint/graph.py Outdated Show resolved Hide resolved
pylint/checkers/refactoring/refactoring_checker.py Outdated Show resolved Hide resolved
pylint/checkers/refactoring/refactoring_checker.py Outdated Show resolved Hide resolved
pylint/checkers/refactoring/refactoring_checker.py Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Jan 16, 2023

Codecov Report

Merging #7611 (5bf97a2) into main (1427461) will decrease coverage by 0.30%.
Report is 6 commits behind head on main.
The diff coverage is 100.00%.

❗ Current head 5bf97a2 differs from pull request most recent head 466a92a. Consider uploading reports for the commit 466a92a to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #7611      +/-   ##
==========================================
- Coverage   95.76%   95.46%   -0.30%     
==========================================
  Files         173      176       +3     
  Lines       18616    18649      +33     
==========================================
- Hits        17827    17803      -24     
- Misses        789      846      +57     
Files Changed Coverage Δ
pylint/checkers/refactoring/refactoring_checker.py 98.21% <100.00%> (-0.02%) ⬇️
pylint/graph.py 91.46% <100.00%> (+5.09%) ⬆️

... and 114 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement ✨ Improvement to a component Needs review 🔍 Needs to be reviewed by one or multiple more persons Needs take over 🛎️ Orignal implementer went away but the code could be salvaged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Check for circular comparisons and other comparison improvements
4 participants