From b4c1e39f0f3c2bd9ae372aeddcd48f366c5306cb Mon Sep 17 00:00:00 2001 From: Ti Chi Robot Date: Wed, 8 Jun 2022 10:40:45 +0800 Subject: [PATCH] system-variables: add system variable tidb_gc_max_wait_time (#8720) (#8813) * add tidb_gc_max_wait_time * Apply suggestions from code review Co-authored-by: Morgan Tocker * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> * add tidb_gc_max_wait_time in GC Configuration * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Aolin Zhang Co-authored-by: Aolin Co-authored-by: Morgan Tocker Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- garbage-collection-configuration.md | 7 +++++++ system-variables.md | 9 +++++++++ 2 files changed, 16 insertions(+) diff --git a/garbage-collection-configuration.md b/garbage-collection-configuration.md index bfdd2e83257ac..7033c47b9fbff 100644 --- a/garbage-collection-configuration.md +++ b/garbage-collection-configuration.md @@ -12,6 +12,7 @@ Garbage collection is configured via the following system variables: * [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time-new-in-v50) * [`tidb_gc_concurrency`](/system-variables.md#tidb_gc_concurrency-new-in-v50) * [`tidb_gc_scan_lock_mode`](/system-variables.md#tidb_gc_scan_lock_mode-new-in-v50) +* [`tidb_gc_max_wait_time`](/system-variables.md#tidb_gc_max_wait_time-new-in-v610) ## GC I/O limit @@ -35,6 +36,12 @@ The `CENTRAL` garbage collection mode is no longer supported. The `DISTRIBUTED` For information on changes in previous releases, refer to earlier versions of this document using the _TIDB version selector_ in the left hand menu. +## Changes in TiDB 6.1.0 + +Before TiDB v6.1.0, the transaction in TiDB does not affect the GC safe point. Since v6.1.0, TiDB considers the startTS of the transaction when calculating the GC safe point, to resolve the problem that the data to be accessed has been cleared. If the transaction is too long, the safe point will be blocked for a long time, which affects the application performance. + +In TiDB v6.1.0, the system variable [`tidb_gc_max_wait_time`](/system-variables.md#tidb_gc_max_wait_time-new-in-v610) is introduced to control the maximum time that active transactions block the GC safe point. After the value is exceeded, the GC safe point is forwarded forcefully. + ### GC in Compaction Filter Based on the `DISTRIBUTED` GC mode, the mechanism of GC in Compaction Filter uses the compaction process of RocksDB, instead of a separate GC worker thread, to run GC. This new GC mechanism helps to avoid extra disk read caused by GC. Also, after clearing the obsolete data, it avoids a large number of left tombstone marks which degrade the sequential scan performance. The following example shows how to enable the mechanism in the TiKV configuration file: diff --git a/system-variables.md b/system-variables.md index d1b28fb52a475..270dc0c313817 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1265,6 +1265,15 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified > - A large amount of history data may affect performance to a certain degree, especially for range queries such as `select count(*) from t` > - If there is any transaction that has been running longer than `tidb_gc_life_time`, during GC, the data since `start_ts` is retained for this transaction to continue execution. For example, if `tidb_gc_life_time` is configured to 10 minutes, among all transactions being executed, the transaction that starts earliest has been running for 15 minutes, GC will retain data of the recent 15 minutes. +### tidb_gc_max_wait_time New in v6.1.0 + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `86400` +- Range: `[600, 31536000]` +- Unit: Seconds +- This variable is used to set the maximum time that active transactions block the GC safe point. During each time of GC, the safe point does not exceed the start time of the ongoing transactions by default. If the runtime of active transactions does not exceed this variable value, the GC safe point will be blocked until the runtime exceeds this value. This variable value is an integer type. + ### tidb_gc_run_interval New in v5.0 - Scope: GLOBAL