Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix cast to decimal overflow bug #3922

Merged
merged 12 commits into from
Jan 26, 2022
Merged

Fix cast to decimal overflow bug #3922

merged 12 commits into from
Jan 26, 2022

Conversation

guo-shaoge
Copy link
Contributor

@guo-shaoge guo-shaoge commented Jan 20, 2022 •
edited by JaySon-Huang
Loading

What problem does this PR solve?

Issue Number: close #3920

Problem Summary: todo

What is changed and how it works?

  1. DAGContext.h/.cpp: expose TiDBSQLFLags so we can use it in unittest(TIDBSQLFlags::OVERFLOW_AS_WARNING). Also fix one clang-tidy.
  2. FunctionsTiDBConversion.h: fix the logic that check if overflow happens or not. We should compare inner_value with inner_max_value or out_value with outer_max_value. I choose inner_value here.
  3. gtest_tidb_conversion.cpp: add unit test when overflow happens.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU: cast(decimal/int/enum/datetime as decimal) will got performance regression
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Fix the issue that casting `INT` to `DECIMAL` might cause overflow

@guo-shaoge
fix cast to decimal overflow check bug
fcff139 
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@guo-shaoge
fix fmt
2fb31e5 
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jan 20, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • fuzhe1989

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jan 20, 2022
@guo-shaoge
add case to check value if overflow
4f717d3 
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@guo-shaoge
Merge branch 'fix_cast_overflow_bug' of github.com:guo-shaoge/tics in…
9e6db1e 
…to fix_cast_overflow_bug

Signed-off-by: guo-shaoge <shaoge1994@163.com>

Conflicts:
	dbms/src/Functions/tests/gtest_tidb_conversion.cpp
@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jan 21, 2022
@guo-shaoge
fix clang-tidy and fmt
270e81f 
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@guo-shaoge
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Jan 21, 2022

Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/1102/cobertura/
(Coverage detail url is limited office network access)

lines: 44.7% (51537 out of 115273)
branches: 7.0% (89870 out of 1275668)

@guo-shaoge
fix comment
ec1d975 
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@guo-shaoge
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Jan 24, 2022

Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/1120/cobertura/
(Coverage detail url is limited office network access)

lines: 44.7% (51544 out of 115280)
branches: 7.0% (89919 out of 1275798)

fuzhe1989
fuzhe1989 reviewed Jan 24, 2022
View reviewed changes
dbms/src/Flash/Coprocessor/DAGContext.h Outdated Show resolved Hide resolved
dbms/src/Flash/Coprocessor/DAGContext.h Outdated Show resolved Hide resolved
dbms/src/Functions/FunctionsTiDBConversion.h Outdated Show resolved Hide resolved
dbms/src/Functions/FunctionsTiDBConversion.h Outdated Show resolved Hide resolved
dbms/src/Functions/FunctionsTiDBConversion.h Outdated
{
context.getDAGContext()->handleOverflowError("cast to decimal", Errors::Types::Truncated);
if (value > 0)
return static_cast<UType>(max_value);
if (outer_value > 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (outer_value > 0)
if (outer_value >= 0)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because scaled_max_value cannot be zero, so this equal doesn't matter.

@guo-shaoge
fix fmt
6220996 
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@ti-chi-bot ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jan 25, 2022
@guo-shaoge
use sub-namespace instead of enum for TiDBSQLFlags
1a06c96 
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@ti-chi-bot ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 25, 2022
@guo-shaoge
fix case
ba104f5 
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@sre-bot
Copy link
Collaborator

sre-bot commented Jan 26, 2022

Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/1184/cobertura/
(Coverage detail url is limited office network access)

lines: 44.7% (51712 out of 115579)
branches: 7.1% (90067 out of 1275637)

@ti-chi-bot ti-chi-bot merged commit 6ea6c80 into pingcap:master Jan 26, 2022
@guo-shaoge guo-shaoge added the needs-cherry-pick-release-5.1 PR which needs to be cherry-picked to release-5.1 label Jan 26, 2022
ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Jan 26, 2022
@guo-shaoge @ti-chi-bot
This is an automated cherry-pick of pingcap#3922
45e55cf 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Jan 26, 2022
Merged
12 tasks
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #3942.

@fuzhe1989 fuzhe1989 deleted the fix_cast_overflow_bug branch January 26, 2022 06:13
ti-chi-bot added a commit that referenced this pull request Jan 27, 2022
@ti-chi-bot
Fix cast to decimal overflow bug (#3922) (#3942)
48a2afc
@guo-shaoge guo-shaoge added the needs-cherry-pick-release-5.3 Type: Need cherry pick to release-5.3 label Feb 21, 2022
ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Feb 21, 2022
@guo-shaoge @ti-chi-bot
This is an automated cherry-pick of pingcap#3922
f55cd5e 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Feb 21, 2022
Merged
12 tasks
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #4073.

@guo-shaoge guo-shaoge added needs-cherry-pick-release-5.0 PR which needs to be cherry-picked to release-5.0 needs-cherry-pick-release-5.2 PR which needs to be cherry-picked to release-5.2 needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. labels Feb 21, 2022
ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Feb 21, 2022
@guo-shaoge @ti-chi-bot
This is an automated cherry-pick of pingcap#3922
57d421e 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Feb 21, 2022
@guo-shaoge @ti-chi-bot
This is an automated cherry-pick of pingcap#3922
75937eb 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Feb 21, 2022
Merged
12 tasks
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #4081.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #4082.

@ti-chi-bot ti-chi-bot mentioned this pull request Feb 21, 2022
Merged
12 tasks
ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Feb 21, 2022
@guo-shaoge @ti-chi-bot
This is an automated cherry-pick of pingcap#3922
2b40910 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Feb 21, 2022
Merged
12 tasks
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #4083.

ti-chi-bot added a commit that referenced this pull request Feb 22, 2022
@ti-chi-bot
Fix cast to decimal overflow bug (#3922) (#4073)
f281880 
close #3920
@sre-bot
Copy link
Collaborator

sre-bot commented Apr 12, 2022

lines: 44.3% (51706 out of 116690)
branches: 7.1% (90052 out of 1277197)

ti-chi-bot added a commit that referenced this pull request Apr 13, 2022
@ti-chi-bot
Fix cast to decimal overflow bug (#3922) (#4081)
6806724 
close #3920
@JaySon-Huang JaySon-Huang added the type/bugfix This PR fixes a bug. label Apr 20, 2022
ti-chi-bot added a commit that referenced this pull request Apr 26, 2022
@ti-chi-bot
Fix cast to decimal overflow bug (#3922) (#4082)
06e5081 
close #3920
ti-chi-bot added a commit that referenced this pull request Jun 16, 2022
@ti-chi-bot
Fix cast to decimal overflow bug (#3922) (#4083)
b25ec50 
close #3920
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fuzhe1989 fuzhe1989 fuzhe1989 approved these changes

Assignees
No one assigned
Labels
needs-cherry-pick-release-5.0 PR which needs to be cherry-picked to release-5.0 needs-cherry-pick-release-5.1 PR which needs to be cherry-picked to release-5.1 needs-cherry-pick-release-5.2 PR which needs to be cherry-picked to release-5.2 needs-cherry-pick-release-5.3 Type: Need cherry pick to release-5.3 needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1. type/bugfix This PR fixes a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

cast int to decimal got wrong result when exceeds range
5 participants
@guo-shaoge @ti-chi-bot @sre-bot @fuzhe1989 @JaySon-Huang