*: support pessimistic transaction (experimental feature) #10297

coocood · 2019-04-28T15:30:26Z

What problem does this PR solve?

Avoid transaction failure caused by write conflict, especially when the transaction cannot be retried automatically.

What is changed and how it works?

Use BEGIN LOCK syntax to start a new pessimistic transaction.
In the pessimistic transaction, write pessimistic lock on INSERT/UPDATE/DELETE/SELECT FOR UPDATE.
Pessimistic locks don't have value, don't block read.
Pessimistic locks blocks writes to make sure there is no write conflict during Prewrite phase.

Check List

Tests

Unit test
Integration test

Code changes

Has exported function/method change
Has exported variable/fields change
Has interface methods change
Has persistent data change

Side effects

Possible performance regression
Increased code complexity

Related changes

Need to update the documentation
Need to be included in the release note

codecov · 2019-04-28T15:41:25Z

Codecov Report

❗ No coverage uploaded for pull request base (master@5617809). Click here to learn what that means.
The diff coverage is 21.8142%.

@@            Coverage Diff             @@
##             master    #10297   +/-   ##
==========================================
  Coverage          ?   77.012%           
==========================================
  Files             ?       412           
  Lines             ?     86232           
  Branches          ?         0           
==========================================
  Hits              ?     66409           
  Misses            ?     14754           
  Partials          ?      5069

coocood · 2019-04-28T15:46:39Z

/run-all-tests

config/config.go

zhouqiang-cl · 2019-04-29T09:41:24Z

/run-all-tests tidb-test=pr/795

config/config.go

config/config.toml.example

executor/adapter.go

tiancaiamao · 2019-04-29T08:45:58Z

session/session_test.go

+
+func (s *testSessionSuite) TestPessimisticTxn(c *C) {
+	if !config.GetGlobalConfig().PessimisticTxn.Enable {
+		c.Skip("pessimistic transaction is not enabled")


The default conf is false, this test will never run?!

session/session_test.go

store/tikv/2pc.go

tiancaiamao · 2019-04-30T07:40:21Z

LGTM

executor/adapter.go

executor/prepared.go

store/tikv/2pc.go

executor/point_get.go

coocood · 2019-05-06T06:44:50Z

@lysu @jackysp PTAL

zhouqiang-cl · 2019-05-06T07:10:00Z

The make dev test takes too long and seems something wrong in this pr

lysu · 2019-05-06T09:35:57Z

store/tikv/2pc.go

@@ -130,7 +160,9 @@ func newTwoPhaseCommitter(txn *tikvTxn, connID uint64) (*twoPhaseCommitter, erro
 			}
 			delCnt++
 		}
-		keys = append(keys, k)
+		if isPessimistic && !bytes.Equal(k, c.primaryKey) {


maybe should be?

Suggested change

if isPessimistic && !bytes.Equal(k, c.primaryKey) {

if !isPessimistic || !bytes.Equal(k, c.primaryKey) {

after dig, maybe old one is ok too, c.primaryKey always be null if isPessimistic=true

ping @coocood

@lysu I know, but add isPessimistic is more explicit that this code is only related to pessimistic lock.

lysu · 2019-05-06T10:59:33Z

rest LGTM for pessimistic disabled logic

store/tikv/txn.go

jackysp · 2019-05-09T08:13:02Z

store/tikv/2pc.go

 	}

 	for _, pair := range txn.assertions {
 		mutation, ok := mutations[string(pair.key)]
 		if !ok {
-			logutil.Logger(context.Background()).Error("ASSERTION FAIL!!! assertion exists but no mutation?",
-				zap.Stringer("assertion", pair))
+			// It's possible when a transaction inserted a key then deleted it later.


@tiancaiamao is this reasonable?

jackysp

rest LGTM

…tic-txn Conflicts: go.mod go.sum store/mockstore/mocktikv/mvcc_leveldb.go

zhouqiang-cl · 2019-05-10T06:27:28Z

make check always hang for very long time

coocood · 2019-05-10T12:00:00Z

/run-all-tests

coocood added 2 commits April 28, 2019 23:16

support pessimistic transaction

5727b48

fix error string and method comment

89c134e

update config

41d79c7

coocood force-pushed the pessimistic-txn branch from b0ef687 to 41d79c7 Compare April 29, 2019 02:44

jackysp added sig/transaction SIG:Transaction type/new-feature labels Apr 29, 2019

jackysp changed the title ~~support pessimistic transaction (experimental feature)~~ *: support pessimistic transaction (experimental feature) Apr 29, 2019

support pessimistic when autocommit = 0, and check unique conflict.

7c241fd

tiancaiamao reviewed Apr 29, 2019

View reviewed changes

config/config.go Outdated Show resolved Hide resolved

tiancaiamao reviewed Apr 29, 2019

View reviewed changes

coocood added 3 commits April 30, 2019 11:14

address comment

3ccbab1

use default

1c454df

use session variable

48f7d90

tiancaiamao added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 30, 2019

youjiali1995 mentioned this pull request May 4, 2019

[WIP] implement pessimistic lock tikv/tikv#4618

Closed

jackysp reviewed May 5, 2019

View reviewed changes

executor/adapter.go Outdated Show resolved Hide resolved

jackysp reviewed May 5, 2019

View reviewed changes

executor/adapter.go Outdated Show resolved Hide resolved

jackysp reviewed May 5, 2019

View reviewed changes

executor/adapter.go Outdated Show resolved Hide resolved

lysu reviewed May 5, 2019

View reviewed changes

executor/adapter.go Outdated Show resolved Hide resolved

executor/adapter.go Show resolved Hide resolved

executor/prepared.go Outdated Show resolved Hide resolved

store/tikv/2pc.go Show resolved Hide resolved

executor/point_get.go Show resolved Hide resolved

address comments

4206bbc

lysu reviewed May 6, 2019

View reviewed changes

jackysp reviewed May 6, 2019

View reviewed changes

store/tikv/txn.go Show resolved Hide resolved

jackysp reviewed May 9, 2019

View reviewed changes

coocood added 2 commits May 10, 2019 13:03

Merge commit '5617809e922e2af62c14899429f69d5a267bec9d' into pessimis…

356c8d1

…tic-txn Conflicts: go.mod go.sum store/mockstore/mocktikv/mvcc_leveldb.go

fix test

a6613b1

coocood added 3 commits May 10, 2019 19:37

fix bug

3233257

fix bug introduced by merge

5ef2732

Merge branch 'master' into pessimistic-txn

6ffa846

jackysp added the status/all tests passed label May 11, 2019

jackysp approved these changes May 11, 2019

View reviewed changes

jackysp merged commit 373748a into pingcap:master May 11, 2019

coocood deleted the pessimistic-txn branch May 13, 2019 04:15

youjiali1995 mentioned this pull request May 15, 2019

*: support pessimistic transaction (experimental feature) tikv/tikv#4698

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: support pessimistic transaction (experimental feature) #10297

*: support pessimistic transaction (experimental feature) #10297

coocood commented Apr 28, 2019

codecov bot commented Apr 28, 2019 •

edited

Loading

coocood commented Apr 28, 2019

zhouqiang-cl commented Apr 29, 2019

tiancaiamao Apr 29, 2019

coocood Apr 30, 2019

tiancaiamao commented Apr 30, 2019

coocood commented May 6, 2019

zhouqiang-cl commented May 6, 2019

lysu May 6, 2019 •

edited

Loading

lysu May 6, 2019

jackysp May 9, 2019

coocood May 9, 2019

lysu commented May 6, 2019

jackysp May 9, 2019

jackysp left a comment

zhouqiang-cl commented May 10, 2019

coocood commented May 10, 2019

	if isPessimistic && !bytes.Equal(k, c.primaryKey) {
	if !isPessimistic \|\| !bytes.Equal(k, c.primaryKey) {

*: support pessimistic transaction (experimental feature) #10297

*: support pessimistic transaction (experimental feature) #10297

Conversation

coocood commented Apr 28, 2019

What problem does this PR solve?

What is changed and how it works?

Check List

codecov bot commented Apr 28, 2019 • edited Loading

Codecov Report

coocood commented Apr 28, 2019

zhouqiang-cl commented Apr 29, 2019

tiancaiamao Apr 29, 2019

Choose a reason for hiding this comment

coocood Apr 30, 2019

Choose a reason for hiding this comment

tiancaiamao commented Apr 30, 2019

coocood commented May 6, 2019

zhouqiang-cl commented May 6, 2019

lysu May 6, 2019 • edited Loading

Choose a reason for hiding this comment

lysu May 6, 2019

Choose a reason for hiding this comment

jackysp May 9, 2019

Choose a reason for hiding this comment

coocood May 9, 2019

Choose a reason for hiding this comment

lysu commented May 6, 2019

jackysp May 9, 2019

Choose a reason for hiding this comment

jackysp left a comment

Choose a reason for hiding this comment

zhouqiang-cl commented May 10, 2019

coocood commented May 10, 2019

codecov bot commented Apr 28, 2019 •

edited

Loading

lysu May 6, 2019 •

edited

Loading