Add information of hashAgg for memory control #6819

wshwsh12 · 2021-08-06T03:10:40Z

First-time contributors' checklist

I've signed Contributor License Agreement that's required for repo owners to accept my contribution.

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions (in Chinese).

What is the related PR or file link(s)?

This PR is translated from:
Other reference link(s):

Do your changes match any of the following descriptions?

Delete files
Change aliases
Need modification after applied to another branch
Might cause conflicts after applied to another branch

ti-chi-bot · 2021-08-06T03:10:41Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

qiancai

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

wshwsh12 · 2021-08-06T03:11:55Z

/cc @XuHuaiyu

TomShawn · 2021-08-06T03:14:30Z

/rebase

configure-memory-usage.md

Liuxiaozhen12

not sure the form of command I suggested is correct or not. Just suggestions.

configure-memory-usage.md

qiancai

@wshwsh12 PTAL

configure-memory-usage.md

qiancai · 2021-08-10T06:33:23Z

configure-memory-usage.md

+- 落盘行为会由参数 [`mem-quota-query`](/tidb-configuration-file.md#mem-quota-query)，[`oom-use-tmp-storage`](/tidb-configuration-file.md#oom-use-tmp-storage)，[`oom-use-tmp-storage`](/tidb-configuration-file.md#oom-use-tmp-storage), [`tmp-storage-path`](/tidb-configuration-file.md#tmp-storage-path),[`tmp-storage-quota`](/tidb-configuration-file.md#tmp-storage-quota) 共同控制。
+- 当落盘被触发时，会在日志中打印一条包含关键字 `memory exceeds quota, spill to disk now` 或 `memory exceeds quota, set aggregate mode to spill-mode` 的日志。
+- Sort，MergeJoin，HashJoin 落盘是从 v4.0.0 版本开始引入的，HashAgg 落盘是从 v5.2.0 版本开始引入的。
+- 含 HashAgg 的 SQL 引起 OOM 时，可以尝试设置 concurrency = 1 来触发落盘。


对于除 HashAgg 外的其他算子，是不是不支持手动设置参数触发落盘呢？

其他算子落盘是默认开启。可以通过 oom-use-tmp-storage 来控制所有的落盘行为。

configure-memory-usage.md

qiancai · 2021-08-13T02:58:13Z

configure-memory-usage.md

+
+## tidb-server 其它内存控制策略
+
+### 流控


这里要不要写成 “动态内存控制功能“

写成”流量控制“ 可以吗？ ”动态内存控制功能“太宽泛，数据落盘也能算动态内存控制功能

configure-memory-usage.md

Co-authored-by: Grace Cai <[email protected]>

configure-memory-usage.md

Co-authored-by: Grace Cai <[email protected]>

qiancai

LGTM

XuHuaiyu

LGTM

ti-chi-bot · 2021-08-13T03:48:39Z

@XuHuaiyu: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

TomShawn · 2021-08-13T06:35:16Z

/remove-status LGT1
/status LGT2

TomShawn · 2021-08-13T06:35:21Z

/merge

ti-chi-bot · 2021-08-13T06:35:23Z

This pull request has been accepted and is ready to merge.

Commit hash: d557021

ti-chi-bot requested a review from TomShawn August 6, 2021 03:10

ti-chi-bot added missing-translation-status This PR does not have translation status info. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 6, 2021

wshwsh12 added the v5.2 This PR/issue applies to TiDB v5.2. label Aug 6, 2021

ti-chi-bot requested a review from XuHuaiyu August 6, 2021 03:11

wshwsh12 added 2 commits August 6, 2021 03:15

tmp

aacce01

docs

72f10e1

github-actions bot force-pushed the 0805 branch from 9d4af40 to 72f10e1 Compare August 6, 2021 03:15

TomShawn added translation/doing This PR’s assignee is translating this PR. and removed missing-translation-status This PR does not have translation status info. labels Aug 6, 2021

TomShawn assigned Liuxiaozhen12 Aug 6, 2021

TomShawn reviewed Aug 9, 2021

View reviewed changes

configure-memory-usage.md Outdated Show resolved Hide resolved

Update configure-memory-usage.md

cb23852

TomShawn added the status/PTAL This PR is ready for reviewing. label Aug 9, 2021

XuHuaiyu reviewed Aug 9, 2021

View reviewed changes

configure-memory-usage.md Outdated Show resolved Hide resolved

Liuxiaozhen12 reviewed Aug 10, 2021

View reviewed changes

configure-memory-usage.md Outdated Show resolved Hide resolved

configure-memory-usage.md Outdated Show resolved Hide resolved

configure-memory-usage.md Outdated Show resolved Hide resolved

configure-memory-usage.md Outdated Show resolved Hide resolved

TomShawn added status/require-change Needs the author to address comments. and removed status/PTAL This PR is ready for reviewing. labels Aug 10, 2021

Liuxiaozhen12 reviewed Aug 10, 2021

View reviewed changes

configure-memory-usage.md Show resolved Hide resolved

wshwsh12 added 2 commits August 10, 2021 11:09

address comments

791f754

address comments

5e445b1

wshwsh12 requested review from XuHuaiyu and Liuxiaozhen12 August 10, 2021 03:13

qiancai reviewed Aug 10, 2021

View reviewed changes

wshwsh12 added 2 commits August 10, 2021 14:52

address comments

e39adba

address comments

de2beca

TomShawn removed the status/require-change Needs the author to address comments. label Aug 10, 2021

address comment

774c060

qiancai reviewed Aug 13, 2021

View reviewed changes

wshwsh12 and others added 9 commits August 13, 2021 11:03

Update configure-memory-usage.md

643e47a

Co-authored-by: Grace Cai <[email protected]>

Update configure-memory-usage.md

da4dcd4

Co-authored-by: Grace Cai <[email protected]>

Update configure-memory-usage.md

3fee95f

Co-authored-by: Grace Cai <[email protected]>

Update configure-memory-usage.md

f89beb2

Co-authored-by: Grace Cai <[email protected]>

Update configure-memory-usage.md

2c022a9

Co-authored-by: Grace Cai <[email protected]>

Update configure-memory-usage.md

0745364

Co-authored-by: Grace Cai <[email protected]>

Update configure-memory-usage.md

aeb6ae8

Co-authored-by: Grace Cai <[email protected]>

address comment

c264f36

address comment

869577d

qiancai reviewed Aug 13, 2021

View reviewed changes

configure-memory-usage.md Show resolved Hide resolved

wshwsh12 and others added 2 commits August 13, 2021 11:28

Update configure-memory-usage.md

70da078

Co-authored-by: Grace Cai <[email protected]>

address comment

d557021

qiancai approved these changes Aug 13, 2021

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 13, 2021

XuHuaiyu reviewed Aug 13, 2021

View reviewed changes

XuHuaiyu approved these changes Aug 13, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 13, 2021

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 13, 2021

ti-chi-bot merged commit 863371c into pingcap:master Aug 13, 2021

Liuxiaozhen12 mentioned this pull request Aug 16, 2021

Add information of hashAgg for memory control pingcap/docs#6171

Merged

11 tasks

Liuxiaozhen12 added translation/done This PR has been translated from English into Chinese and updated to pingcap/docs-cn in a PR. and removed translation/doing This PR’s assignee is translating this PR. labels Aug 16, 2021

TomShawn mentioned this pull request Jun 27, 2022

add Other strategies for memory control #10159

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add information of hashAgg for memory control #6819

Add information of hashAgg for memory control #6819

wshwsh12 commented Aug 6, 2021

ti-chi-bot commented Aug 6, 2021 •

edited

Loading

wshwsh12 commented Aug 6, 2021

TomShawn commented Aug 6, 2021

Liuxiaozhen12 left a comment

qiancai left a comment

qiancai Aug 10, 2021

wshwsh12 Aug 10, 2021

qiancai Aug 13, 2021

wshwsh12 Aug 13, 2021

qiancai left a comment

XuHuaiyu left a comment

ti-chi-bot commented Aug 13, 2021

TomShawn commented Aug 13, 2021

TomShawn commented Aug 13, 2021

ti-chi-bot commented Aug 13, 2021

Add information of hashAgg for memory control #6819

Add information of hashAgg for memory control #6819

Conversation

wshwsh12 commented Aug 6, 2021

First-time contributors' checklist

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

What is the related PR or file link(s)?

Do your changes match any of the following descriptions?

ti-chi-bot commented Aug 6, 2021 • edited Loading

wshwsh12 commented Aug 6, 2021

TomShawn commented Aug 6, 2021

Liuxiaozhen12 left a comment

Choose a reason for hiding this comment

qiancai left a comment

Choose a reason for hiding this comment

qiancai Aug 10, 2021

Choose a reason for hiding this comment

wshwsh12 Aug 10, 2021

Choose a reason for hiding this comment

qiancai Aug 13, 2021

Choose a reason for hiding this comment

wshwsh12 Aug 13, 2021

Choose a reason for hiding this comment

qiancai left a comment

Choose a reason for hiding this comment

XuHuaiyu left a comment

Choose a reason for hiding this comment

ti-chi-bot commented Aug 13, 2021

TomShawn commented Aug 13, 2021

TomShawn commented Aug 13, 2021

ti-chi-bot commented Aug 13, 2021

ti-chi-bot commented Aug 6, 2021 •

edited

Loading