Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pump/: Fix get L0 files num and trigger vlog.gcTS step by step #648

Merged
merged 6 commits into from
Jun 26, 2019

Conversation

july2993
Copy link
Contributor

@july2993 july2993 commented Jun 23, 2019

What problem does this PR solve?

Refine gc of pump

What is changed and how it works?

1, Add a API to trigger gc
2, Check L0 files by GetProperty instead of Stats
Stats can't get the right L0 files num, see:
https://github.com/syndtr/goleveldb/pull/283/files
3, call vlog.gcTS step by step, help free space quickly
4, update goleveldb dependency
only one new commit
syndtr/goleveldb@02440ea
Fix the stat about levels of the Stats API

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    call debug/gc/trigger

Code changes

Side effects

Related changes

  • Need to cherry-pick to the release branch
  • Need to be included in the release note

1, Add a API to trigger gc
2, Check L0 files by *GetProperty* instead of Stats
Stats can't get the right L0 files num, see:
https://github.com/syndtr/goleveldb/pull/283/files
3, call vlog.gcTS step by step, help free space quickly
@july2993
Copy link
Contributor Author

/run-all-tests

pump/storage/storage.go Outdated Show resolved Hide resolved
pump/storage/storage.go Outdated Show resolved Hide resolved
pump/storage/storage.go Outdated Show resolved Hide resolved
pump/server.go Outdated Show resolved Hide resolved
pump/server.go Show resolved Hide resolved
@july2993 july2993 changed the title pump/: Refine gc of pump pump/: Fix get L0 files num and trigger vlog.gcTS step by step Jun 24, 2019
Copy link
Contributor

@suzaku suzaku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@suzaku
Copy link
Contributor

suzaku commented Jun 26, 2019

/run-all-tests

@@ -654,6 +663,10 @@ func (a *Append) doGCTS(ts int64) {
}
break
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add log for kv gc and vlog gc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add in 5e91e87

@IANTHEREAL
Copy link
Collaborator

Rest LGTM

@IANTHEREAL
Copy link
Collaborator

LGTM

@july2993
Copy link
Contributor Author

/run-all-tests

@july2993 july2993 merged commit 061e321 into pingcap:master Jun 26, 2019
@july2993 july2993 deleted the hjh/gc_trigger branch June 26, 2019 08:20
july2993 added a commit to july2993/tidb-binlog that referenced this pull request Jul 1, 2019
…ap#648)

1, Add a API to trigger gc
2, Check L0 files by GetProperty instead of Stats
Stats can't get the right L0 files num, see:
https://github.com/syndtr/goleveldb/pull/283/files
3, call vlog.gcTS step by step, help free space quickly
4, update goleveldb dependency
only one new commit
syndtr/goleveldb@02440ea
Fix the stat about levels of the Stats API
july2993 added a commit that referenced this pull request Jul 2, 2019
…#658)

1, Add a API to trigger gc
2, Check L0 files by GetProperty instead of Stats
Stats can't get the right L0 files num, see:
https://github.com/syndtr/goleveldb/pull/283/files
3, call vlog.gcTS step by step, help free space quickly
4, update goleveldb dependency
only one new commit
syndtr/goleveldb@02440ea
Fix the stat about levels of the Stats API
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants