Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Checker Be Parallel #7706

Open
nolouch opened this issue Jan 15, 2024 · 0 comments
Open

Make Checker Be Parallel #7706

nolouch opened this issue Jan 15, 2024 · 0 comments
Labels
type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@nolouch
Copy link
Contributor

nolouch commented Jan 15, 2024

Background

Currently, Many scheduling logic based on the Checkers, such as failover, rules, and so on. but the checker is not the same as a scheduler, it scans the regions from start to end, and then does some calculation logic. it may spend much time each round if there are many regions and stores. such as:
image
image

The checker will be bottleneck because it cannot parallel.

Proposal

Consider making the framework can be parallel with an argument, like concurrency = 2

Before:
image

After:

image
or
image

other consider:

  • scan interval can be automatically tune
  • the checker can be like a scheduler, can add and remove. also, we can define the order of the checker with some interface. it helps some scenarios to prioritize to do some checker. what's more, the rule checker can also be split into smaller checker, like the offline stores, but mostly CPU usage in moveBetterLocaltion, we can disable moveBetterLocaltion, only do offline checks.
  • also RegionRuleCache will help speed up the checker fit the regions.
  • concurrency can be tuned by the user, better can be controlled under specified CPU usage.
@nolouch nolouch added type/development The issue belongs to a development tasks type/feature-request Categorizes issue or PR as related to a new feature. and removed type/development The issue belongs to a development tasks labels Jan 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

1 participant