Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance heartbeat processing capabilities #7897

Closed
7 tasks done
nolouch opened this issue Mar 7, 2024 · 0 comments
Closed
7 tasks done

Enhance heartbeat processing capabilities #7897

nolouch opened this issue Mar 7, 2024 · 0 comments
Labels
type/development The issue belongs to a development tasks

Comments

@nolouch
Copy link
Contributor

nolouch commented Mar 7, 2024 •
edited
Loading

Development Task

Sigle node processing capacity

Improve the diagnosis process and implement hierarchical processing for Heartbeat logic

Optimize Lock and Reduce lock holding time

Improve the memory alloc

Scalability

  • [TBD] Proposal
    • only metadata(lookup) part, statics part not consider
@nolouch nolouch added the type/development The issue belongs to a development tasks label Mar 7, 2024
@nolouch nolouch mentioned this issue Mar 7, 2024
Closed
This was referenced Apr 3, 2024
Merged
Merged
ti-chi-bot bot added a commit that referenced this issue Apr 8, 2024
@nolouch @ti-chi-bot
pkg/schedule: optimize the lock usage of operator controller (#8032)
949d602 
ref #7897

pkg/schedule: optimize the lock usage of the operator controller
- use sync.Map for operators, which is friendly for check operators on the heartbeat hot path
- reduce the lock hold time by splitting the locks.

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot added a commit that referenced this issue Apr 9, 2024
@nolouch @ti-chi-bot
pkg/ratelimit: introduce an executor that can run with a limiter (#8024)
35e8e95 
ref #7897

pkg/ratelimit: Introduce an executor that can run with a limiter
- async runner can run the task with the limiter, the task will parallel run.
- sync runner used to keep the behavior like before

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
This was referenced Apr 10, 2024
Merged
Merged
Merged
ti-chi-bot bot added a commit that referenced this issue Apr 11, 2024
@nolouch @ti-chi-bot
pkg/schedule: put merge operators together (#8050)
96a69fc 
ref #7897, close #8049

pkg/schedule: put merge operators together to maintain atomicity

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@ti-chi-bot ti-chi-bot mentioned this issue Apr 15, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Apr 15, 2024
@ti-chi-bot @nolouch
pkg/schedule: put merge operators together (#8050) (#8063)
3ec92bd 
ref #7897, close #8049

pkg/schedule: put merge operators together to maintain atomicity

Signed-off-by: nolouch <[email protected]>

Co-authored-by: nolouch <[email protected]>
ti-chi-bot bot pushed a commit that referenced this issue Apr 15, 2024
@nolouch
*: optimize heartbeat process with concurrent runner - part 1 (#8053)
0214778 
ref #7897

*: Optimize the heartbeat process with the concurrent runner 
- add a switch about the runner
- move some tasks out, not block the heartbeat process

Signed-off-by: nolouch <[email protected]>
@rleungx rleungx mentioned this issue Apr 22, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue Apr 24, 2024
@nolouch @ti-chi-bot
*: optimize heartbeat process with concurrent runner - part 2 (#8052)
141186e 
ref #7897

Optimize heartbeat process
- Split the statics updates on the subtree

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@nolouch nolouch mentioned this issue Apr 28, 2024
Merged
@rleungx rleungx pinned this issue Apr 29, 2024
@rleungx rleungx mentioned this issue Apr 29, 2024
Merged
@nolouch nolouch mentioned this issue Apr 29, 2024
Merged
nolouch added a commit to nolouch/pd that referenced this issue Apr 29, 2024
@nolouch @ti-chi-bot
*: optimize heartbeat process with concurrent runner - part 2 (tikv#8052
cd0e1bf 
)

ref tikv#7897

Optimize heartbeat process
- Split the statics updates on the subtree

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot added a commit that referenced this issue Apr 30, 2024
@rleungx @ti-chi-bot
pkg: fix RegionStatsNeedUpdate condition (#8133)
8ee18a4 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@nolouch nolouch mentioned this issue Apr 30, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue May 6, 2024
@nolouch @ti-chi-bot
core: check stats healthy by reference (#8132)
f83feba 
ref #7897, close #8131

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot added a commit that referenced this issue May 7, 2024
@rleungx @ti-chi-bot
*: split into multiple runner for heartbeat (#8130)
a3c5950 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@rleungx rleungx mentioned this issue May 9, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 10, 2024
@nolouch
*: add pending task metrics in runner and add templ (#8138)
78cb081 
ref #7897

Signed-off-by: nolouch <[email protected]>
This was referenced May 10, 2024
Merged
Merged
ti-chi-bot bot added a commit that referenced this issue May 13, 2024
@rleungx @ti-chi-bot
*: remove task name opt (#8168)
2278609 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@rleungx rleungx mentioned this issue May 15, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue May 17, 2024
@JmPotato @ti-chi-bot @HuSharp
core/region: use a separate tree to check for overlaps in the subtree (…
36a1fa2 
…#8185)

ref #7897, close #8136

Previously, removing overlapped regions in subtrees required finding overlaps for each subtree individually.
This PR introduces a separate tree specifically for checking overlaps in the subtrees, thereby reducing lookup times and improving performance.

Signed-off-by: JmPotato <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Co-authored-by: Hu# <[email protected]>
@rleungx rleungx mentioned this issue May 17, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 18, 2024
@nolouch
core/region: fix sub tree may not consist with root tree (#8187)
6ded6f8 
ref #7897

core/region: fix the sub tree may not consist of root tree

Signed-off-by: nolouch <[email protected]>
@JmPotato JmPotato mentioned this issue May 20, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 21, 2024
@JmPotato
core/region: fix the bug causing incorrect reference count after a ca…
19eda09 
…che reset (#8201)

ref #7897

#8132 introduced a count reference to ensure the consistency between
the root tree and subtrees, however, the reference count was not handled correctly during the cache
reset process, which may lead to incorrect reference counts in later heartbeat processing.

Signed-off-by: JmPotato <[email protected]>
@JmPotato JmPotato mentioned this issue May 21, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 21, 2024
@rleungx
pkg: reduce the allocation of observe (#8188)
7731ee4 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>
ti-chi-bot bot pushed a commit that referenced this issue May 21, 2024
@rleungx
*: use a separate runner for updating subtree (#8158)
58e7580 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>
@nolouch nolouch mentioned this issue May 22, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 22, 2024
@nolouch
pkg/statistics: reduce the memory alloc for hot statistics (#8208)
43e9492 
ref #7897, close #8207

pkg/statistics: reduce the memory alloc for hot statistics

Signed-off-by: nolouch <[email protected]>
@nolouch nolouch mentioned this issue May 22, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue May 22, 2024
@nolouch
metrics: update grafana template to add heartbeat latency overview (#…
b871b57 
…8209)

ref #7897

metrics: update grafana template to add heartbeat latency overview

Signed-off-by: nolouch <[email protected]>
ti-chi-bot bot added a commit that referenced this issue May 22, 2024
@JmPotato @ti-chi-bot
core/region: optimize the efficiency of random regions selecting (#8205)
0056569 
ref #7897

Optimize the efficiency of random regions selecting.

Signed-off-by: JmPotato <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@nolouch nolouch mentioned this issue May 22, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue May 23, 2024
@nolouch @ti-chi-bot
metrics: fix the duplicate avg metrics (#8210)
6b8d227 
ref #7897

metrics: fix the duplicate avg metrics

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@JmPotato JmPotato mentioned this issue May 23, 2024
Merged
okJiang pushed a commit to okJiang/pd that referenced this issue May 23, 2024
@nolouch @ti-chi-bot @okJiang
metrics: fix the duplicate avg metrics (tikv#8210)
ec6e1b5 
ref tikv#7897

metrics: fix the duplicate avg metrics

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Signed-off-by: okJiang <[email protected]>
okJiang pushed a commit to okJiang/pd that referenced this issue May 23, 2024
@nolouch @ti-chi-bot @okJiang
metrics: fix the duplicate avg metrics (tikv#8210)
27a74a5 
ref tikv#7897

metrics: fix the duplicate avg metrics

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Signed-off-by: okJiang <[email protected]>
ti-chi-bot bot pushed a commit that referenced this issue May 23, 2024
@JmPotato
core/region: optimize the logic of randomly selecting regions within …
4cd42b3 
…a single range (#8211)

ref #7897

In the case of only one key range, follow the fast path logic to avoid unnecessary random and loop operations.

Signed-off-by: JmPotato <[email protected]>
@rleungx rleungx mentioned this issue May 24, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue May 28, 2024
@rleungx @ti-chi-bot
*: batch process peer task (#8213)
9d580d0 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot added a commit that referenced this issue May 28, 2024
@rleungx @ti-chi-bot
*: refactor store info (#6830)
5eb66e0 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
This was referenced May 31, 2024
Merged
Open
ti-chi-bot bot added a commit that referenced this issue Jun 13, 2024
@rleungx @ti-chi-bot
pkg: remove old duplicated task (#8234)
bf02fb5 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot added a commit that referenced this issue Jun 13, 2024
@rleungx @ti-chi-bot
*: optimize memory usage (#8164)
3e34b80 
ref #7897

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@nolouch nolouch closed this as completed Aug 1, 2024
@JmPotato JmPotato unpinned this issue Aug 6, 2024
@HuSharp HuSharp mentioned this issue Aug 22, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue Aug 22, 2024
@okJiang @ti-chi-bot
heartbeat: reduce unnecessary RunTask (#8559)
84138e4 
ref #7897

Signed-off-by: okJiang <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/development The issue belongs to a development tasks
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nolouch