Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schema_storage: fix schema GC threshold & improve memory management #3172

Merged
merged 34 commits into from
Nov 4, 2021
Merged
Changes from 2 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
2a41b00
schema_storage: make runtime GC snapshots in time
liuzix Oct 29, 2021
96d31ec
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Oct 29, 2021
28aa013
processor: remove schema GC lag
liuzix Oct 29, 2021
34b0512
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Oct 29, 2021
3a6fe19
processor: fix schema GC TS
liuzix Oct 29, 2021
db6eb2d
Merge branch 'zixiong-fix-schema-gc' of github.com:liuzix/ticdc into …
liuzix Oct 29, 2021
1898bf6
processor: fix schema GC TS
liuzix Oct 29, 2021
4fb7b0f
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Oct 29, 2021
c3d1ad5
add test
liuzix Nov 2, 2021
20fb58e
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 2, 2021
4a60654
adjust code
liuzix Nov 2, 2021
3a57fd5
Merge branch 'zixiong-fix-schema-gc' of github.com:liuzix/ticdc into …
liuzix Nov 2, 2021
e2902d6
fix bug
liuzix Nov 2, 2021
848b1fa
fix ut
liuzix Nov 2, 2021
388f25c
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 2, 2021
2448247
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 2, 2021
465c218
add metrics for GC Ts
liuzix Nov 2, 2021
ad4d71d
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 2, 2021
dd24ed9
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 3, 2021
3c8898f
fix metrics
liuzix Nov 3, 2021
c588cc0
add Grafana panel
liuzix Nov 3, 2021
697a497
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 3, 2021
0c6ebb5
fix Grafana panel
liuzix Nov 3, 2021
e462f4e
Merge branch 'zixiong-fix-schema-gc' of github.com:liuzix/ticdc into …
liuzix Nov 3, 2021
0092257
fix update processor lastSchemaTs
liuzix Nov 3, 2021
0d8458d
fix Grafana panel instant = true
liuzix Nov 3, 2021
5c10065
add metrics prefix
liuzix Nov 3, 2021
797bd05
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 3, 2021
a7127e6
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 3, 2021
e274e64
add nil check for changefeed status
liuzix Nov 3, 2021
d50d33a
Merge branch 'zixiong-fix-schema-gc' of github.com:liuzix/ticdc into …
liuzix Nov 3, 2021
16fe2f5
fix Grafana pluginVersion
liuzix Nov 4, 2021
082a66c
fix Grafana value name
liuzix Nov 4, 2021
3105724
Merge branch 'master' into zixiong-fix-schema-gc
liuzix Nov 4, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions cdc/entry/schema_storage.go
Original file line number Diff line number Diff line change
Expand Up @@ -806,6 +806,11 @@ func (s *schemaStorageImpl) DoGC(ts uint64) {
s.snaps[i].PrintStatus(log.Debug)
}
}
// Sets the pointers to nil so that the snaps can be
// GC'd by Go runtime in time.
for i := 0; i < startIdx; i++ {
s.snaps[i] = nil
}
s.snaps = s.snaps[startIdx:]
liuzix marked this conversation as resolved.
Show resolved Hide resolved
atomic.StoreUint64(&s.gcTs, s.snaps[0].currentTs)
log.Info("finished gc in schema storage", zap.Uint64("gcTs", s.snaps[0].currentTs))
Expand Down