Add more metrics about PageStorage #6175

JaySon-Huang · 2022-10-22T10:16:09Z

What problem does this PR solve?

Issue Number: ref #6162

Problem Summary:

What is changed and how it works?

System table changes

Add max_pack_count_in_delta to system.dt_tables
Remove outdated columns for PageStorage V2 in system.dt_tables

Grafana changes

http://172.16.5.85:21540/d/qa-OjKNVk/test-cluster-tiflash-summary?orgId=1&from=1666781469235&to=1666803740418

Rename "MVCC Snapshots" -> "Read Snapshots"
Rename "StoragePool" -> "Page"
Show log-files disk usage and log-files number
Add metrics about P99 of write duration
Breakdown the GC duration into a smaller granularity
Move the Page GC panels from "Storage" to "Page"
Show the number of pages in memory

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

ti-chi-bot · 2022-10-22T10:16:10Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

breezewish
lidezhu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

dbms/src/Storages/Page/V3/PageStorageImpl.cpp

JaySon-Huang · 2022-11-03T10:52:09Z

~~Need update the tiflash-summary.json~~

Updated

JaySon-Huang · 2022-11-07T07:58:35Z

Working: the avg and max column files count per segment in the delta layer

Update: tiflash changes done, need another PR for tidb system table

JaySon-Huang · 2022-11-08T05:59:34Z

/run-all-tests

lidezhu

LGTM

breezewish · 2022-11-08T07:14:46Z

dbms/src/Common/TiFlashMetrics.h

@@ -149,17 +149,31 @@ namespace DB
        F(type_flush, {{"type", "flush"}}, ExpBuckets{0.001, 2, 20}),                                                                     \
        F(type_split, {{"type", "split"}}, ExpBuckets{0.001, 2, 20}))                                                                     \
    M(tiflash_storage_page_gc_count, "Total number of page's gc execution.", Counter,                                                     \
-        F(type_exec, {"type", "exec"}),                                                                                                   \
-        F(type_low_write, {"type", "low_write"}),                                                                                         \
+        F(type_v2, {"type", "exec"}),                                                                                                     \


Is there a reason not changing the label value?

I thought we could get a smoother line before and after the upgrade. But changing the label value seems more reasonable.
Changed.

JaySon-Huang · 2022-11-09T04:39:56Z

/merge

ti-chi-bot · 2022-11-09T04:39:57Z

@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

JaySon-Huang · 2022-11-09T04:43:27Z

/merge cancel

metrics/grafana/tiflash_summary.json

JaySon-Huang · 2022-11-09T04:45:25Z

/merge

ti-chi-bot · 2022-11-09T04:45:26Z

@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-11-09T04:45:28Z

This pull request has been accepted and is ready to merge.

Commit hash: 20f14a0

ti-chi-bot · 2022-11-09T04:46:00Z

@JaySon-Huang: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

JaySon-Huang commented Oct 22, 2022

View reviewed changes

dbms/src/Storages/Page/V3/PageStorageImpl.cpp Outdated Show resolved Hide resolved

JaySon-Huang mentioned this pull request Oct 26, 2022

PageStorage: Fix peak memory usage when running GC on PageDirectory #6168

Merged

12 tasks

ti-chi-bot removed the do-not-merge/needs-linked-issue label Oct 26, 2022

JaySon-Huang force-pushed the add_ps_latency branch from 3aadff7 to 3050bee Compare October 26, 2022 10:21

JaySon-Huang mentioned this pull request Nov 3, 2022

testing: pageworkload HeavySkewWriteRead support deleting data #6178

Merged

12 tasks

JaySon-Huang force-pushed the add_ps_latency branch from c0c16b8 to 3873c70 Compare November 4, 2022 03:36

JaySon-Huang changed the title ~~[WIP] Add more metrics about PageStorage~~ Add more metrics about PageStorage Nov 4, 2022

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 4, 2022

JaySon-Huang added 9 commits November 7, 2022 15:38

Add breakdown metrics about ps gc

1a2c3ef

Add breakdown metrics about ps write

32d8048

Add latency metrics about ps reading

c9188c8

More precise bucket

afdde18

Add panels

42a6e20

Update metrics

2094c01

Keep track of log-file disk usage

c18f2ac

Remove read metrics

f410b6e

Update grafana

d727ee3

JaySon-Huang force-pushed the add_ps_latency branch from 7ef00cc to d727ee3 Compare November 7, 2022 07:39

Update metrics

cb17b2e

JaySon-Huang added 4 commits November 7, 2022 16:12

Format files

9ed683f

Remove outdate stat for ps v2

0249156

Add max column file num per segment in delta

0d3cf4f

Update comments

d566f3d

ti-chi-bot requested review from flowbehappy and lidezhu November 7, 2022 10:33

Merge remote-tracking branch 'upstream/master' into add_ps_latency

f8146dd

lidezhu approved these changes Nov 8, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Nov 8, 2022

JaySon-Huang added 2 commits November 8, 2022 16:22

Change the position of max_pack_count_in_delta

a48c9d9

Update the result of status

5af0962

JaySon-Huang mentioned this pull request Nov 9, 2022

systable: Adapt with tiflash_tables changes pingcap/tidb#39001

Merged

12 tasks

breezewish approved these changes Nov 9, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Nov 9, 2022

Update the metrics label

4ec70bf

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 9, 2022

Merge branch 'master' into add_ps_latency

5c920e1

ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Nov 9, 2022

JaySon-Huang commented Nov 9, 2022

View reviewed changes

metrics/grafana/tiflash_summary.json Outdated Show resolved Hide resolved

Update metrics/grafana/tiflash_summary.json

20f14a0

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 9, 2022

Merge branch 'master' into add_ps_latency

0086931

ti-chi-bot merged commit dc19413 into pingcap:master Nov 9, 2022

JaySon-Huang deleted the add_ps_latency branch November 9, 2022 06:09

JaySon-Huang mentioned this pull request Nov 20, 2022

Duplicate Grafana panel id #6334

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add more metrics about PageStorage #6175

Add more metrics about PageStorage #6175

JaySon-Huang commented Oct 22, 2022 •

edited

Loading

ti-chi-bot commented Oct 22, 2022 •

edited

Loading

JaySon-Huang commented Nov 3, 2022 •

edited

Loading

JaySon-Huang commented Nov 7, 2022 •

edited

Loading

JaySon-Huang commented Nov 8, 2022

lidezhu left a comment

breezewish Nov 8, 2022

JaySon-Huang Nov 9, 2022

JaySon-Huang commented Nov 9, 2022

ti-chi-bot commented Nov 9, 2022

JaySon-Huang commented Nov 9, 2022

JaySon-Huang commented Nov 9, 2022

ti-chi-bot commented Nov 9, 2022

ti-chi-bot commented Nov 9, 2022

ti-chi-bot commented Nov 9, 2022

Add more metrics about PageStorage #6175

Add more metrics about PageStorage #6175

Conversation

JaySon-Huang commented Oct 22, 2022 • edited Loading

What problem does this PR solve?

What is changed and how it works?

System table changes

Grafana changes

Check List

Release note

ti-chi-bot commented Oct 22, 2022 • edited Loading

JaySon-Huang commented Nov 3, 2022 • edited Loading

JaySon-Huang commented Nov 7, 2022 • edited Loading

JaySon-Huang commented Nov 8, 2022

lidezhu left a comment

Choose a reason for hiding this comment

breezewish Nov 8, 2022

Choose a reason for hiding this comment

JaySon-Huang Nov 9, 2022

Choose a reason for hiding this comment

JaySon-Huang commented Nov 9, 2022

ti-chi-bot commented Nov 9, 2022

JaySon-Huang commented Nov 9, 2022

JaySon-Huang commented Nov 9, 2022

ti-chi-bot commented Nov 9, 2022

ti-chi-bot commented Nov 9, 2022

ti-chi-bot commented Nov 9, 2022

JaySon-Huang commented Oct 22, 2022 •

edited

Loading

ti-chi-bot commented Oct 22, 2022 •

edited

Loading

JaySon-Huang commented Nov 3, 2022 •

edited

Loading

JaySon-Huang commented Nov 7, 2022 •

edited

Loading