Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After restoring data with BR, the split regions will not result in a large number of merges, thus avoiding unreasonable splits. #55057

Closed
shaoxiqian opened this issue Jul 30, 2024 · 1 comment
Assignees
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@shaoxiqian
Copy link
Contributor

Enhancement

BR nightly resotre 50,000 databases, 250,000 tables.
After the restore is completed, the number of regions on the TiKV side is as follows.
image
pd region healty
image

in house testing we found that both v1 and v2 see about 35~40% drop in QPS when schemas increase from 1 to 50001. The cause of the performance regression stems from the generation of a vast number of regions.

After merging regions, under the nightly v2 mode, the performance with 50001 schemas actually shows a slight improvement compared to having just 1 schema. But even with the schedule.merge-schedule-limit set to 512, it almost takes 24 hours to complete the merge of regions. The time cost is substantial.

Therefore, we hope that after restoring data with BR, the split regions will not generate a large number of merges, thus avoiding unreasonable splits.

@shaoxiqian shaoxiqian added the type/enhancement The issue or PR belongs to an enhancement. label Jul 30, 2024
@shaoxiqian shaoxiqian assigned shaoxiqian and Leavrth and unassigned shaoxiqian Jul 30, 2024
@Leavrth
Copy link
Contributor

Leavrth commented Aug 12, 2024

related issue #53532

@Leavrth Leavrth closed this as completed Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

No branches or pull requests

2 participants