Skip to content

Commit

Permalink
Update the default GC scan lock mode back to LEGACY (#5136) (#5146)
Browse files Browse the repository at this point in the history
  • Loading branch information
ti-srebot authored Mar 29, 2021
1 parent 9a72ad5 commit 5a98a32
Show file tree
Hide file tree
Showing 10 changed files with 29 additions and 39 deletions.
19 changes: 0 additions & 19 deletions alert-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -605,25 +605,6 @@ This section gives the alert rules for the TiKV component.

The pressure on the apply Raft log thread is too high. It is often caused by a burst of writes.

#### `TiDB_tikvclient_gc_action_fail` (only happen when in special configurations)

* Alert rule:

`sum(increase(tidb_tikvclient_gc_action_result{type="fail”}[1m])) > 10`

> **Note:**
>
> In TiDB 3.* versions, the `tidb_tikvclient_gc_action_result` metric exists but does not have a value. It's because distributed garbage collection (GC) is introduced in the TiDB 3.0 version but will not be performed in TiDB.
* Description:

There are many Regions where GC fails to work.

* Solution:

1. It is normally because the GC concurrency is set too high. You can moderately lower the GC concurrency degree, and you need to first confirm that the failed GC is caused by the busy server.
2. You can moderately lower the concurrency degree by adjusting [`tikv_db_concurrency`](/system-variables.md#tidb_gc_concurrency).

### Warning-level alerts

#### `TiKV_leader_drops`
Expand Down
2 changes: 1 addition & 1 deletion error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the

* Error Number: 8055

The current snapshot is too old. The data may have been garbage collected. You can increase the value of [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time) to avoid this problem. The new version of TiDB automatically reserves data for long-running transactions. Usually this error does not occur.
The current snapshot is too old. The data may have been garbage collected. You can increase the value of [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time-new-in-v50-ga) to avoid this problem. The new version of TiDB automatically reserves data for long-running transactions. Usually this error does not occur.

See [garbage collection overview](/garbage-collection-overview.md) and [garbage collection configuration](/garbage-collection-configuration.md).

Expand Down
4 changes: 1 addition & 3 deletions faq/sql-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,9 +134,7 @@ Deleting a large amount of data leaves a lot of useless keys, affecting the quer

## What should I do if it is slow to reclaim storage space after deleting data?

Because TiDB uses Multiversion concurrency control (MVCC), deleting data does not immediately reclaim space. Garbage collection is delayed so that concurrent transactions are able to see earlier versions of rows. This can be configured via the [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time) (default: `10m0s`) system variable.

When performing a backup, the `tidb_gc_life_time` is also automatically extended so that the backup can complete successfully.
Because TiDB uses Multiversion concurrency control (MVCC), deleting data does not immediately reclaim space. Garbage collection is delayed so that concurrent transactions are able to see earlier versions of rows. This can be configured via the [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time-new-in-v50-ga) (default: `10m0s`) system variable.

## Does `SHOW PROCESSLIST` display the system process ID?

Expand Down
2 changes: 1 addition & 1 deletion faq/tidb-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ The accessed Region is not available. A Raft Group is not available, with possib

#### ERROR 9006 (HY000): GC life time is shorter than transaction duration

The interval of `GC Life Time` is too short. The data that should have been read by long transactions might be deleted. You can adjust [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time) using the following command:
The interval of `GC Life Time` is too short. The data that should have been read by long transactions might be deleted. You can adjust [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time-new-in-v50-ga) using the following command:

{{< copyable "sql" >}}

Expand Down
10 changes: 5 additions & 5 deletions garbage-collection-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ summary: Learn about GC configuration parameters.

Garbage collection is configured via the following system variables:

* [`tidb_gc_enable`](/system-variables.md#tidb_gc_enable)
* [`tidb_gc_run_interval`](/system-variables.md#tidb_gc_run_interval)
* [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time)
* [`tidb_gc_concurrency`](/system-variables.md#tidb_gc_concurrency)
* [`tidb_gc_scan_lock_mode`](/system-variables.md#tidb_gc_scan_lock_mode)
* [`tidb_gc_enable`](/system-variables.md#tidb_gc_enable-new-in-v50-ga)
* [`tidb_gc_run_interval`](/system-variables.md#tidb_gc_run_interval-new-in-v50-ga)
* [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time-new-in-v50-ga)
* [`tidb_gc_concurrency`](/system-variables.md#tidb_gc_concurrency-new-in-v50-ga)
* [`tidb_gc_scan_lock_mode`](/system-variables.md#tidb_gc_scan_lock_mode-new-in-v50-ga)

## GC I/O limit

Expand Down
9 changes: 8 additions & 1 deletion garbage-collection-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,14 @@ The TiDB transaction model is implemented based on [Google's Percolator](https:/

The Resolve Locks step clears the locks before the safe point. This means that if the primary key of a lock is committed, this lock needs to be committed; otherwise, it needs to be rolled back. If the primary key is still locked (not committed or rolled back), this transaction is seen as timing out and rolled back.

By default, TiDB will bypass the Raft layer and directly scans data on each TiKV node. This is configurable via the system variable [`tidb_gc_scan_lock_mode`](/system-variables.md#tidb_gc_scan_lock_mode). In the previous default (`LEGACY`), the GC leader sends requests to all Regions to scan obsolete locks, checks the primary key statuses of scanned locks, and sends requests to commit or roll back the corresponding transaction.
The Resolve Locks step is implemented in either of the following two ways, which can be configured using the system variable [`tidb_gc_scan_lock_mode`](/system-variables.md#tidb_gc_scan_lock_mode-new-in-v50-ga):

> **Warning:**
>
> Currently, `PHYSICAL` (Green GC) is an experimental feature. It is not recommended that you use it in the production environment.
- `LEGACY` (default): The GC leader sends requests to all Regions to scan obsolete locks, checks the primary key statuses of scanned locks, and sends requests to commit or roll back the corresponding transaction.
- `PHYSICAL`: TiDB bypasses the Raft layer and directly scans data on each TiKV node.

### Delete Ranges

Expand Down
2 changes: 1 addition & 1 deletion read-historical-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ In TiDB, Garbage Collection (GC) runs periodically to remove the obsolete data v

Pay special attention to the following:

- [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time): This system variable is used to configure the retention time of earlier modifications (default: `10m0s`).
- [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time-new-in-v50-ga): This system variable is used to configure the retention time of earlier modifications (default: `10m0s`).
- The output of `SELECT * FROM mysql.tidb WHERE variable_name = 'tikv_gc_safe_point'`. This is the current `safePoint` where you can read historical data up to. It is updated every time the garbage collection process is run.

## Example
Expand Down
2 changes: 1 addition & 1 deletion sql-statements/sql-statement-flashback-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ summary: Learn how to recover tables using the `FLASHBACK TABLE` statement.

The `FLASHBACK TABLE` syntax is introduced since TiDB 4.0. You can use the `FLASHBACK TABLE` statement to restore the tables and data dropped by the `DROP` or `TRUNCATE` operation within the Garbage Collection (GC) lifetime.

The system variable [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time) (default: `10m0s`) defines the retention time of earlier versions of rows. The current `safePoint` of where garabage collection has been performed up to can be obtained with the following query:
The system variable [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time-new-in-v50-ga) (default: `10m0s`) defines the retention time of earlier versions of rows. The current `safePoint` of where garabage collection has been performed up to can be obtained with the following query:

{{< copyable "sql" >}}

Expand Down
16 changes: 10 additions & 6 deletions system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -617,19 +617,19 @@ For a system upgraded to v5.0.0-rc from an earlier version, if you have not modi
- This variable is used to change the default priority for statements executed on a TiDB server. A use case is to ensure that a particular user that is performing OLAP queries receives lower priority than users performing OLTP queries.
- You can set the value of this variable to `NO_PRIORITY`, `LOW_PRIORITY`, `DELAYED` or `HIGH_PRIORITY`.

### tidb_gc_concurrency
### tidb_gc_concurrency <span class="version-mark">New in v5.0 GA</span>

- Scope: GLOBAL
- Default: -1
- Specifies the number of threads in the [Resolve Locks](/garbage-collection-overview.md#resolve-locks) step of GC. A value of `-1` means that TiDB will automatically decide the number of garbage collection threads to use.

### tidb_gc_enable
### tidb_gc_enable <span class="version-mark">New in v5.0 GA</span>

- Scope: GLOBAL
- Default value: ON
- Enables garbage collection for TiKV. Disabling garbage collection will reduce system performance, as old versions of rows will no longer be purged.

## tidb_gc_life_time
## tidb_gc_life_time <span class="version-mark">New in v5.0 GA</span>

- Scope: GLOBAL
- Default: `"10m0s"`
Expand All @@ -642,16 +642,20 @@ For a system upgraded to v5.0.0-rc from an earlier version, if you have not modi
> - A large amount of history data may affect performance to a certain degree, especially for range queries such as `select count(*) from t`
> - If there is any transaction that has been running longer than `tidb_gc_life_time`, during GC, the data since `start_ts` is retained for this transaction to continue execution. For example, if `tidb_gc_life_time` is configured to 10 minutes, among all transactions being executed, the transaction that starts earliest has been running for 15 minutes, GC will retain data of the recent 15 minutes.

### tidb_gc_run_interval
### tidb_gc_run_interval <span class="version-mark">New in v5.0 GA</span>

- Scope: GLOBAL
- Default value: `"10m0s"`
- Specifies the GC interval, in the format of Go Duration, for example, `"1h30m"`, and `"15m"`

### tidb_gc_scan_lock_mode
### tidb_gc_scan_lock_mode <span class="version-mark">New in v5.0 GA</span>

> **Warning:**
>
> Currently, Green GC is an experimental feature. It is not recommended that you use it in the production environment.

- Scope: GLOBAL
- Default value: `PHYSICAL`
- Default value: `LEGACY`
- Possible values:
- `LEGACY`: Uses the old way of scanning, that is, disable Green GC.
- `PHYSICAL`: Uses the physical scanning method, that is, enable Green GC.
Expand Down
2 changes: 1 addition & 1 deletion tidb-troubleshooting-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -554,7 +554,7 @@ Check the specific cause for busy by viewing the monitor **Grafana** -> **TiKV**

The transaction duration exceeds the GC lifetime (10 minutes by default).

You can increase the GC lifetime by modifying the [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time) system variable. Generally, it is not recommended to modify this parameter, because changing it might cause many old versions to pile up if this transaction has a large number of `UPDATE` and `DELETE` statements.
You can increase the GC lifetime by modifying the [`tidb_gc_life_time`](/system-variables.md#tidb_gc_life_time-new-in-v50-ga) system variable. Generally, it is not recommended to modify this parameter, because changing it might cause many old versions to pile up if this transaction has a large number of `UPDATE` and `DELETE` statements.

- 7.1.2 `txn takes too much time`.

Expand Down

0 comments on commit 5a98a32

Please sign in to comment.