Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The replica count of default placement rule stays constant after replication.max-replicas is updated #3886

Closed
lilinghai opened this issue Jul 19, 2021 · 4 comments · Fixed by #3892
Assignees
Labels
severity/major type/bug The issue is confirmed as a bug.

Comments

@lilinghai
Copy link

Bug Report

What did you do?

deploy tidb cluster with default config
enable placement rules
online update replication.max-replicas to 1
show placement rule

What did you expect to see?

The replica count of default placement rule is 3. This is a little confusing whether I complete the update of replicas count.

What did you see instead?

The replica count of default placement rule stays same with the config replication.max-replicas.

What version of PD are you using (pd-server -V)?

v5.1.0

@lilinghai lilinghai added the type/bug The issue is confirmed as a bug. label Jul 19, 2021
@disksing disksing self-assigned this Jul 19, 2021
@disksing
Copy link
Contributor

The cause is that RuleManager.GetRule does not return a copy of rule but a pointer to the rule which is stored in rule manager. In server.go when we update the rule we are updating the rule in rule manager at the same time. When the rule is about to save to storage, it will be skipped because if we compare the new rule and the rule in rule manager, they are exact the same.

Taken together, the phenomenon is that rule will be temporarily updated. But when the PD is restarted or a leader re-election occurs, the original configuration appears again.

Way to fix: Change methods such as GetRule to return a cloned rule.

@adamotonete
Copy link

When changing the replication.max-replicas to anything different than 1 the service is dying and the client is returning:
Failed to set config: [500] PANIC: runtime error: invalid memory address or nil pointer dereference goroutine 24385 [running]:

Branch master (v5.2.0-alpha-27-gc1f31284)

@disksing
Copy link
Contributor

disksing commented Jul 20, 2021

Hi @adamotonete, thanks for your feedback. I have tried by did not reproduce the panic issue. Do you have more detailed steps to reproduce it?

@mayjiang0203
Copy link

/severity Major

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants