Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: tiny refactor code to reduce txn conflict on 'table_cache_meta' #32387

Merged
merged 7 commits into from
Mar 1, 2022

Conversation

tiancaiamao
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #32386

Problem Summary:

The warning log is caused by the fact that "select ... for update" can't retry when using optimistic transaction.

What is changed and how it works?

Tiny code refactor. Two changes mainly:

  • Remove 'for update', just use the non-'for update' statement

There is no need to worry about write skew abnormity here, we can just use the default RR (SI)

  • Refactor the renew lease code

In the past, I make a goroutine along with a channel for the renew lease operation,
i.e. send the renew operation (func) teller the worker to execute it.

type renewLeaseCh chan func()
go do.renewLease()

func (do *Domain) renewLease() {
	defer func() {
		do.wg.Done()
		logutil.BgLogger().Info("renew lease goroutine exited.")
	}()
	for {
		select {
		case <-do.exit:
			close(do.renewLeaseCh)
			return
		case op := <-do.renewLeaseCh:
			op()
		}
	}
}

And this pattern to avoid duplicated message: https://github.com/pingcap/tidb/pull/31475/files

This is complex than it should be.

To avoid duplicated message, a chan(1) can be used as a token limiter, and spawn a new goroutine is handy:

c.renewReadLease = make(chan struct{}, 1)

select {
case c.renewReadLease <- struct{}{}:
	go c.renewLease(ts, data, leaseDuration)
default:
}

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Tiny refactor

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Feb 15, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • lcwangchao
  • tangenta

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 15, 2022
@tiancaiamao
Copy link
Contributor Author

/run-unit-test

@sre-bot
Copy link
Contributor

sre-bot commented Feb 16, 2022

@tiancaiamao
Copy link
Contributor Author

Ref #25293

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 17, 2022
table/tables/cache.go Outdated Show resolved Hide resolved
@tangenta
Copy link
Contributor

Please make sure this is expected:

Previously each domain can have at most 10 goroutines for renewing lease. After this PR, each cached table in infoSchema can spawn a new goroutine to renew lease.

@tiancaiamao
Copy link
Contributor Author

Please make sure this is expected:

Previously each domain can have at most 10 goroutines for renewing lease. After this PR, each cached table in infoSchema can spawn a new goroutine to renew lease.

This is by design.
In the old code, there will be duplicated renew lease operation, because there are 10 background goroutines, they can handle several message simultaneously.
Now there will be only one active goroutine.

sync.RWMutex
lockingForRead bool
}
lockingForRead tokenLimit
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be a semaphore, why not just use semaphore in go lib?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with semaphore, the channel is more simple for me here.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 1, 2022
@tiancaiamao
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: bb591be

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 1, 2022
@ti-chi-bot
Copy link
Member

@tiancaiamao: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@tiancaiamao
Copy link
Contributor Author

/run-unit-test

@tiancaiamao
Copy link
Contributor Author

/run-mysql-test

@ti-chi-bot ti-chi-bot merged commit d7d6afc into pingcap:master Mar 1, 2022
@tiancaiamao tiancaiamao deleted the reduce-conflict branch March 1, 2022 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Transaction conflict on internal SQL, "mysql.table_cache_meta" table
5 participants