Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workerpool: remove mergeContext #2201

Merged
merged 7 commits into from
Aug 10, 2021

Conversation

liuzix
Copy link
Contributor

@liuzix liuzix commented Jul 1, 2021

What problem does this PR solve?

  • mergeContext spawns a goroutine when Done() is called for the first time. Since we generate a mergeContext for each event before calling its handler, when concurrency is high, we will be running a large number of goroutines concurrently, and these goroutines do little that's useful.

Close #2211

What is changed and how it works?

  • Remove mergeContext completely. Event handler calls can only be cancelled with the ctx used to add that event. They will no longer be cancelled when the workerpool is cancelled.

Check List

Tests

  • Unit test
  • Integration test

Related changes

  • Need to cherry-pick to the release branch

Release note

  • Optimize workerpool for fewer goroutines when concurrency is high.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jul 1, 2021

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • amyangfei
  • overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot requested review from zier-one and lonng July 1, 2021 08:33
@liuzix liuzix added the subject/performance Denotes an issue or pull request is related to replication performance. label Jul 1, 2021
@ti-chi-bot ti-chi-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jul 1, 2021
@liuzix liuzix added needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. status/ptal Could you please take a look? labels Jul 1, 2021
@liuzix
Copy link
Contributor Author

liuzix commented Jul 1, 2021

/run-integration-tests

@liuzix
Copy link
Contributor Author

liuzix commented Jul 1, 2021

/run-leak-tests

pkg/workerpool/pool.go Outdated Show resolved Hide resolved
@liuzix
Copy link
Contributor Author

liuzix commented Jul 2, 2021

/run-leak-tests

@liuzix
Copy link
Contributor Author

liuzix commented Jul 2, 2021

/run-leak-tests

@liuzix
Copy link
Contributor Author

liuzix commented Jul 2, 2021

/run-integration-tests

})
}

time.Sleep(5 * time.Second)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to sleep 5s here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just wanted to make sure that the internal queue in the pool has been fully congested. I can reduce it to 2s.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can loop and check.

for {
  sleep(100ms)
  if check_full? {
    break
  }
  if wait_time > 5s {
    panic("too long")
  }
} 

Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

@overvenus
Copy link
Member

overvenus commented Aug 2, 2021

Could you move this issue forward? It have been tested for a while, not issue found.

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 6, 2021
@amyangfei
Copy link
Contributor

/run-unit-tests
/run-integration-tests

@amyangfei amyangfei added the needs-cherry-pick-release-5.2 Should cherry pick this PR to release-5.2 branch. label Aug 9, 2021
@amyangfei
Copy link
Contributor

/run-integration-tests

3 similar comments
@overvenus
Copy link
Member

/run-integration-tests

@amyangfei
Copy link
Contributor

/run-integration-tests

@amyangfei
Copy link
Contributor

/run-integration-tests

@amyangfei
Copy link
Contributor

/run-integration-tests /tikv=release-5.2

@codecov-commenter
Copy link

Codecov Report

Merging #2201 (bb5cbe2) into master (5a4568b) will decrease coverage by 1.3415%.
The diff coverage is 40.0838%.

@@               Coverage Diff                @@
##             master      #2201        +/-   ##
================================================
- Coverage   57.0620%   55.7204%   -1.3416%     
================================================
  Files           165        168         +3     
  Lines         19435      20418       +983     
================================================
+ Hits          11090      11377       +287     
- Misses         7288       7938       +650     
- Partials       1057       1103        +46     

@ti-chi-bot ti-chi-bot merged commit 632a8bc into pingcap:master Aug 10, 2021
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2486.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2487.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2488.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: cannot checkout 5.2: error checking out 5.2: exit status 1. output: error: pathspec '5.2' did not match any file(s) known to git

@amyangfei
Copy link
Contributor

/run-cherry-picker

amyangfei pushed a commit to amyangfei/tiflow that referenced this pull request Aug 10, 2021
@amyangfei
Copy link
Contributor

/cherry-pick release-5.2

@ti-chi-bot
Copy link
Member

@amyangfei: new pull request created: #2490.

In response to this:

/cherry-pick release-5.2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. needs-cherry-pick-release-5.2 Should cherry pick this PR to release-5.2 branch. release-blocker This issue blocks a release. Please solve it ASAP. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. status/ptal Could you please take a look? subject/performance Denotes an issue or pull request is related to replication performance.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

workerpool.(*mergedContext).Done creates too many goroutines
6 participants