Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Not enough replicas" caused keyspace group split to fail randomly #6550

Closed
binshi-bing opened this issue Jun 1, 2023 · 1 comment · Fixed by #6555
Closed

"Not enough replicas" caused keyspace group split to fail randomly #6550

binshi-bing opened this issue Jun 1, 2023 · 1 comment · Fixed by #6555
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@binshi-bing
Copy link
Contributor

Enhancement Task

What did you do?

QA team ran keyspace group split with pd-ctl

What did you expect to see?

keyspace group split should successfully complete

What did you see instead?

Randomly failed because of "Not enough replicas"

What version of PD are you using (pd-server -V)?

tidbcloud/pd-cse release-6.6-keyspace 9e1e2de

@rleungx
Copy link
Member

rleungx commented Jun 5, 2023

This is because the watcher is started before the bootstrap process. The alloc node loop exits since there is no group in etcd (introduced by #6390). When the bootstrap process is trying to init the default group, it cannot be assigned the members, so it will persist the null to etcd which triggers the above problem.

@ti-chi-bot ti-chi-bot bot closed this as completed in #6555 Jun 5, 2023
ti-chi-bot bot pushed a commit that referenced this issue Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants