expression: fix incorrect result of logical operators when isTrue/ isFalse function is pushed down

[Reminiscent]

What problem does this PR solve?

Close: #15718 #15987
Related PR: #12173
Need to review TiPB176 first
Related to #13962

What is changed and how it works?

See an example of this comment
Before this PR:

MySQL [test]> select a and b as c, a, b from tt order by c limit 4;
+---+------+------+
| c | a    | b    |
+---+------+------+
| NULL |    7 | NULL |
| NULL | NULL | 2    |
| 0 |    0 | 2    |
| 0 | NULL | w    |
+---+------+------+
4 rows in set, 4 warnings (0.00 sec)

MySQL [test]> select a and b as c, a, b from tt order by c;
+---+------+------+
| c | a    | b    |
+---+------+------+
| NULL | NULL | NULL |
| NULL |    7 | NULL |
| NULL | NULL | 2    |
| NULL | NULL | 2    |
| 0 | NULL | w    |
| 0 |    0 | 2    |
| 0 |    7 | f    |
| 1 |    7 | 3    |
+---+------+------+
8 rows in set, 2 warnings (0.01 sec)

After this PR:

mysql> create table tt(a decimal(10, 0), b varchar(1));
Query OK, 0 rows affected (0.02 sec)

mysql> insert into tt values(0, '2'), (7, null), (7, '3'), (NULL, 'w'), (NULL, '2'), (NULL, '2'), (NULL, NULL), (7, 'f');
Query OK, 8 rows affected (0.00 sec)
Records: 8  Duplicates: 0  Warnings: 0

mysql> select a and b as c, a, b from tt order by c limit 4;
+---+------+------+
| c | a    | b    |
+---+------+------+
| NULL | NULL | NULL |
| NULL | NULL | 2    |
| NULL | NULL | 2    |
| NULL |    7 | NULL |
+---+------+------+
4 rows in set, 2 warnings (0.01 sec)

mysql> select a and b as c, a, b from tt order by c;
+---+------+------+
| c | a    | b    |
+---+------+------+
| NULL | NULL | NULL |
| NULL |    7 | NULL |
| NULL | NULL | 2    |
| NULL | NULL | 2    |
| 0 | NULL | w    |
| 0 |    0 | 2    |
| 0 |    7 | f    |
| 1 |    7 | 3    |
+---+------+------+
8 rows in set, 2 warnings (0.00 sec)

The cause of this problem is that when the order by expression in the topN executor is logicalAnd or logicalOr, keepNull should be set to true when the topN executor is pushed down.

Check List

Tests

• Unit test
• Integration test

[codecov]

Codecov Report

Merging #15926 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #15926   +/-   ##
===========================================
  Coverage   80.3437%   80.3437%           
===========================================
  Files           506        506           
  Lines        136618     136618           
===========================================
  Hits         109764     109764           
  Misses        18276      18276           
  Partials       8578       8578           

[XuHuaiyu]

LGTM

[wshwsh12]

LGTM

[XuHuaiyu]

/merge

[sre-bot]

Your auto merge job has been accepted, waiting for 15907, 15845, 15758, 15956, 15776, 15889

[sre-bot]

/run-all-tests

[sre-bot]

@Reminiscent merge failed.

[XuHuaiyu]

@Reminiscent CI failed, please check it.

[XuHuaiyu]

Is this expectant?

[2020-04-16T06:33:09.000Z] FAIL: const_test.go:180: testMySQLConstSuite.TestHighNotPrecedenceMode

[2020-04-16T06:33:09.000Z] 

[2020-04-16T06:33:09.000Z] const_test.go:191:

[2020-04-16T06:33:09.000Z]     r = tk.MustQuery(`SELECT * FROM t1 WHERE NOT a BETWEEN 2 AND 3;`)

[2020-04-16T06:33:09.000Z] /home/jenkins/agent/workspace/tidb_ghpr_check_2@2/go/src/github.com/pingcap/tidb/util/testkit/testkit.go:288:

[2020-04-16T06:33:09.000Z]     tk.c.Check(err, check.IsNil, comment)

[2020-04-16T06:33:09.000Z] ... value *errors.fundamental = runtime error: invalid memory address or nil pointer dereference ("runtime error: invalid memory address or nil pointer dereference")

[2020-04-16T06:33:09.000Z] ... sql:SELECT * FROM t1 WHERE NOT a BETWEEN 2 AND 3;, args:[]

[XuHuaiyu]

[2020-04-16T09:05:14.887Z] PANIC: distsql_builtin_test.go:159: testEvalSuite.TestEval

[2020-04-16T09:05:14.887Z] 

[2020-04-16T09:05:14.887Z] ... Panic: interface conversion: proto.Message is nil, not *tipb.InUnionMetadata (PC=0xFAA711)

[2020-04-16T09:05:14.887Z] 

[2020-04-16T09:05:14.887Z] /usr/local/go/src/runtime/panic.go:679

[2020-04-16T09:05:14.887Z]   in gopanic

[2020-04-16T09:05:14.887Z] /usr/local/go/src/runtime/iface.go:255

[2020-04-16T09:05:14.887Z]   in panicdottypeE

[2020-04-16T09:05:14.887Z] /usr/local/go/src/runtime/iface.go:265

[2020-04-16T09:05:14.887Z]   in panicdottypeI

[2020-04-16T09:05:14.887Z] distsql_builtin.go:74

[2020-04-16T09:05:14.887Z]   in getSignatureByPB

[2020-04-16T09:05:14.887Z] distsql_builtin.go:1056

[2020-04-16T09:05:14.887Z]   in newDistSQLFunctionBySig

[2020-04-16T09:05:14.887Z] distsql_builtin.go:1138

[2020-04-16T09:05:14.887Z]   in PBToExpr

[2020-04-16T09:05:14.887Z] distsql_builtin_test.go:856

[2020-04-16T09:05:14.887Z]   in testEvalSuite.TestEval

[2020-04-16T09:05:14.887Z] /usr/local/go/src/reflect/value.go:321

[2020-04-16T09:05:14.887Z]   in Value.Call

[2020-04-16T09:05:14.887Z] /home/jenkins/agent/workspace/tidb_ghpr_check_2@2/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850

[2020-04-16T09:05:14.887Z]   in suiteRunner.forkTest.func1

[2020-04-16T09:05:14.887Z] /home/jenkins/agent/workspace/tidb_ghpr_check_2@2/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739

[2020-04-16T09:05:14.887Z]   in suiteRunner.forkCall.func1

[2020-04-16T09:05:14.887Z] /usr/local/go/src/runtime/asm_amd64.s:1357

[2020-04-16T09:05:14.887Z]   in goexit

[Reminiscent]

@XuHuaiyu @breeswish PTAL. Thanks!

[XuHuaiyu]

LGTM

[XuHuaiyu]

/merge

[sre-bot]

/run-all-tests

[sre-bot]

/run-all-tests

[sre-bot]

@Reminiscent merge failed.

[breezewish]

Do we need to fix at TiKV side as well? Otherwise the results will be different so that copr-test will fail.

[Reminiscent]

After an offline discussion, we decided to use another method to solve this problem. We will close this PR and reopen a new one.