Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lightning: make OpLevelOptional suppress the error of DoChecksum #45486

Merged
merged 9 commits into from
Aug 7, 2023
2 changes: 2 additions & 0 deletions br/pkg/lightning/importer/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,8 @@ go_library(
"@com_github_tikv_pd_client//:client",
"@io_etcd_go_etcd_client_v3//:client",
"@org_golang_google_grpc//:grpc",
"@org_golang_google_grpc//codes",
"@org_golang_google_grpc//status",
"@org_golang_x_exp//maps",
"@org_golang_x_exp//slices",
"@org_golang_x_sync//errgroup",
Expand Down
26 changes: 20 additions & 6 deletions br/pkg/lightning/importer/table_import.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@
"go.uber.org/multierr"
"go.uber.org/zap"
"golang.org/x/exp/slices"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)

// TableImporter is a helper struct to import a table.
Expand Down Expand Up @@ -1034,15 +1036,27 @@

var remoteChecksum *local.RemoteChecksum
remoteChecksum, err = DoChecksum(ctx, tr.tableInfo)
failpoint.Inject("checksum-error", func() {
tr.logger.Info("failpoint checksum-error injected.")
remoteChecksum = nil
err = status.Error(codes.Unknown, "Checksum meets error.")
lance6716 marked this conversation as resolved.
Show resolved Hide resolved
})

Check warning on line 1043 in br/pkg/lightning/importer/table_import.go

View check run for this annotation

Codecov / codecov/patch

br/pkg/lightning/importer/table_import.go#L1039-L1043

Added lines #L1039 - L1043 were not covered by tests
if err != nil {
if rc.cfg.PostRestore.Checksum == config.OpLevelOptional {
tr.logger.Warn("do checksum failed, will skip this error and go on", log.ShortError(err))
err = nil
}
} else {

Check warning on line 1049 in br/pkg/lightning/importer/table_import.go

View check run for this annotation

Codecov / codecov/patch

br/pkg/lightning/importer/table_import.go#L1045-L1049

Added lines #L1045 - L1049 were not covered by tests
return false, err
}
err = tr.compareChecksum(remoteChecksum, localChecksum)
// with post restore level 'optional', we will skip checksum error
if rc.cfg.PostRestore.Checksum == config.OpLevelOptional {
if err != nil {
tr.logger.Warn("compare checksum failed, will skip this error and go on", log.ShortError(err))
err = nil
if remoteChecksum != nil {
err = tr.compareChecksum(remoteChecksum, localChecksum)
// with post restore level 'optional', we will skip checksum error
if rc.cfg.PostRestore.Checksum == config.OpLevelOptional {
if err != nil {
tr.logger.Warn("compare checksum failed, will skip this error and go on", log.ShortError(err))
err = nil
}

Check warning on line 1059 in br/pkg/lightning/importer/table_import.go

View check run for this annotation

Codecov / codecov/patch

br/pkg/lightning/importer/table_import.go#L1052-L1059

Added lines #L1052 - L1059 were not covered by tests
}
}
} else {
Expand Down
3 changes: 3 additions & 0 deletions br/tests/lightning_routes/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,6 @@ schema-pattern = "routes_a*"
table-pattern = "t*"
target-schema = "routes_b"
target-table = "u"

[post-restore]
checksum = "optional"
5 changes: 5 additions & 0 deletions br/tests/lightning_routes/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,17 @@

set -eux

echo "testing checksum-error..."
export GO_FAILPOINTS="github.com/pingcap/tidb/br/pkg/lightning/importer/checksum-error=1*return()"

run_sql 'DROP DATABASE IF EXISTS routes_a0;'
run_sql 'DROP DATABASE IF EXISTS routes_a1;'
run_sql 'DROP DATABASE IF EXISTS routes_b;'

run_lightning

echo "test checksum-error success!"

run_sql 'SELECT count(1), sum(x) FROM routes_b.u;'
check_contains 'count(1): 4'
check_contains 'sum(x): 259'
Expand Down
Loading