Skip to content

Commit

Permalink
update garbage collection docs
Browse files Browse the repository at this point in the history
  • Loading branch information
morgo committed Dec 30, 2020
1 parent 89fc7dc commit 3b87cac
Show file tree
Hide file tree
Showing 12 changed files with 38 additions and 45 deletions.
4 changes: 2 additions & 2 deletions alert-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -445,7 +445,7 @@ Emergency-level alerts are often caused by a service or node failure. Manual int

* Solution:

1. Perform `select VARIABLE_VALUE from mysql.tidb where VARIABLE_NAME = "tikv_gc_leader_desc"` to locate the `tidb-server` corresponding to the GC leader;
1. Perform `SELECT VARIABLE_VALUE FROM mysql.tidb WHERE VARIABLE_NAME = "tikv_gc_leader_desc"` to locate the `tidb-server` corresponding to the GC leader;
2. View the log of the `tidb-server`, and grep gc_worker tidb.log;
3. If you find that the GC worker has been resolving locks (the last log is "start resolve locks") or deleting ranges (the last log is “start delete {number} ranges”) during this time, it means the GC process is running normally. Otherwise, contact [[email protected]](mailto:[email protected]) to resolve this issue.

Expand Down Expand Up @@ -633,7 +633,7 @@ For the critical-level alerts, a close watch on the abnormal metrics is required
* Solution:

1. It is normally because the GC concurrency is set too high. You can moderately lower the GC concurrency degree, and you need to first confirm that the failed GC is caused by the busy server.
2. You can moderately lower the concurrency degree by running `update set VARIABLE_VALUE="{number}” where VARIABLE_NAME=”tikv_gc_concurrency`.
2. You can moderately lower the concurrency degree by adjusting [`tikv_gc_concurrency`](/system-variables.md#tikv_gc_concurrency).

### Warning-level alerts

Expand Down
18 changes: 9 additions & 9 deletions backup-and-restore-using-dumpling-lightning.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,30 +61,30 @@ The steps to manually modify the GC time are as follows:
{{< copyable "sql" >}}

```sql
SELECT * FROM mysql.tidb WHERE VARIABLE_NAME = 'tikv_gc_life_time';
SHOW GLOBAL VARIABLES LIKE 'tikv_gc_life_time';
```

```sql
+-----------------------+------------------------------------------------------------------------------------------------+
| VARIABLE_NAME | VARIABLE_VALUE |
+-----------------------+------------------------------------------------------------------------------------------------+
| tikv_gc_life_time | 10m0s |
+-----------------------+------------------------------------------------------------------------------------------------+
1 rows in set (0.02 sec)
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| tikv_gc_life_time | 10m0s |
+-------------------+-------+
1 row in set (0.03 sec)
```

{{< copyable "sql" >}}

```sql
UPDATE mysql.tidb SET VARIABLE_VALUE = '720h' WHERE VARIABLE_NAME = 'tikv_gc_life_time';
SET GLOBAL tikv_gc_life_time = '720h';
```

2. After executing the `dumpling` command, restore the GC value of the TiDB cluster to the initial value in step 1:

{{< copyable "sql" >}}

```sql
UPDATE mysql.tidb SET VARIABLE_VALUE = '10m' WHERE VARIABLE_NAME = 'tikv_gc_life_time';
SET GLOBAL tikv_gc_life_time = '10m';
```

## Restore data into TiDB
Expand Down
7 changes: 3 additions & 4 deletions br/backup-and-restore-tool.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,18 +195,17 @@ Each of the above three sub-commands might still include the following three sub

To back up the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table.

If the BR version is earlier than v4.0.3, and the backup duration might exceed the [`tikv_gc_life_time`](/garbage-collection-configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default (`10m0s` means 10 minutes), increase the value of this configuration item.
If the BR version is earlier than v4.0.3, and the backup duration might exceed the [`tikv_gc_life_time`](/system-variables.md#tikv_gc_life_time) configuration which is `10m0s` by default (`10m0s` means 10 minutes), increase the value of this configuration item.

For example, set `tikv_gc_life_time` to `720h`:

{{< copyable "sql" >}}

```sql
mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \
"update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'";
SET GLOBAL tikv_gc_life_time = '720h';
```

Since v4.0.3, BR automatically adapts to GC and you do not need to manually adjust the `tikv_gc_life_time` value.
Since v4.0.3, BR automatically adapts to GC and you do not need to manually adjust the [`tikv_gc_life_time`](/system-variables.md#tikv_gc_life_time) value.

### Back up all the cluster data

Expand Down
8 changes: 4 additions & 4 deletions br/backup-and-restore-use-cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,21 +74,21 @@ It is recommended that you use a network disk to back up and restore data. This

For the detailed usage of the `br backup` command, refer to [BR command-line description](/br/backup-and-restore-tool.md#command-line-description).

1. Before executing the `br backup` command, check the value of the [`tikv_gc_life_time`](/garbage-collection-configuration.md#tikv_gc_life_time) configuration item, and adjust the value appropriately in the MySQL client to make sure that [Garbage Collection](/garbage-collection-overview.md) (GC) does not run during the backup operation.
1. Before executing the `br backup` command, check the value of the [`tikv_gc_life_time`](/system-variables.md#tikv_gc_life_time) configuration item, and adjust the value appropriately in the MySQL client to make sure that [Garbage Collection](/garbage-collection-overview.md) (GC) does not run during the backup operation:

{{< copyable "sql" >}}

```sql
SELECT * FROM mysql.tidb WHERE VARIABLE_NAME = 'tikv_gc_life_time';
UPDATE mysql.tidb SET VARIABLE_VALUE = '720h' WHERE VARIABLE_NAME = 'tikv_gc_life_time';
SHOW GLOBAL VARIABLES LIKE 'tikv_gc_life_time';
SET GLOBAL tikv_gc_life_time = '720h';
```

2. After the backup operation, set the parameter back to the original value.

{{< copyable "sql" >}}

```sql
UPDATE mysql.tidb SET VARIABLE_VALUE = '10m' WHERE VARIABLE_NAME = 'tikv_gc_life_time';
SET GLOBAL tikv_gc_life_time = '10m';
```

> **Note:**
Expand Down
4 changes: 2 additions & 2 deletions dumpling-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,15 +307,15 @@ In other scenarios, if the data size is very large, to avoid export failure due
{{< copyable "sql" >}}

```sql
update mysql.tidb set VARIABLE_VALUE = '720h' where VARIABLE_NAME = 'tikv_gc_life_time';
SET GLOBAL tikv_gc_life_time = '720h';
```

After your operation is completed, set the GC time back (the default value is `10m`):

{{< copyable "sql" >}}

```sql
update mysql.tidb set VARIABLE_VALUE = '10m' where VARIABLE_NAME = 'tikv_gc_life_time';
SET GLOBAL tikv_gc_life_time = '10m';
```

Finally, all the exported data can be imported back to TiDB using [Lightning](/tidb-lightning/tidb-lightning-backends.md).
Expand Down
10 changes: 2 additions & 8 deletions error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ In addition, TiDB has the following unique error codes:

An unsupported database isolation level is set.

If you cannot modify the codes because you are using a third-party tool or framework, consider using `tidb_skip_isolation_level_check` to bypass this check.
If you cannot modify the codes because you are using a third-party tool or framework, consider using [`tidb_skip_isolation_level_check`](/system-variables.md#tidb_skip_isolation_level_check) to bypass this check.

{{< copyable "sql" >}}

Expand Down Expand Up @@ -179,16 +179,10 @@ In addition, TiDB has the following unique error codes:

* Error Number: 8055

The current snapshot is too old. The data may have been garbage collected. You can increase the value of `tikv_gc_life_time` to avoid this problem. The new version of TiDB automatically reserves data for long-running transactions. Usually this error does not occur.
The current snapshot is too old. The data may have been garbage collected. You can increase the value of [`tikv_gc_life_time`](/system-variables.md#tikv_gc_life_time) to avoid this problem. The new version of TiDB automatically reserves data for long-running transactions. Usually this error does not occur.

See [garbage collection overview](/garbage-collection-overview.md) and [garbage collection configuration](/garbage-collection-configuration.md).

{{< copyable "sql" >}}

```sql
update mysql.tidb set VARIABLE_VALUE="24h" where VARIABLE_NAME="tikv_gc_life_time";
```

* Error Number: 8059

The auto-random ID is exhausted and cannot be allocated. There is no way to recover from such errors currently. It is recommended to use bigint when using the auto random feature to obtain the maximum number of assignment. And try to avoid manually assigning values to the auto random column.
Expand Down
6 changes: 3 additions & 3 deletions faq/migration-tidb-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ TiDB can provide services while Loader is running because Loader inserts the dat

### How to export the data in TiDB?

Currently, TiDB does not support `select into outfile`. You can use the following methods to export the data in TiDB:
You can use the following methods to export the data in TiDB:

- See [MySQL uses mysqldump to export part of the table data](https://blog.csdn.net/xin_yu_xin/article/details/7574662) in Chinese and export data using mysqldump and the `WHERE` clause.
- Use the MySQL client to export the results of `select` to a file.
Expand Down Expand Up @@ -158,5 +158,5 @@ You can configure concurrent GC to increase the speed of reclaiming storage spac
{{< copyable "sql" >}}

```sql
update mysql.tidb set VARIABLE_VALUE="3" where VARIABLE_NAME="tikv_gc_concurrency";
```
SET GLOBAL tikv_gc_concurrency = 3;
```
2 changes: 1 addition & 1 deletion faq/sql-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ You can configure concurrent GC to increase the speed of reclaiming storage spac
{{< copyable "sql" >}}

```sql
update mysql.tidb set VARIABLE_VALUE="3" where VARIABLE_NAME="tikv_gc_concurrency";
SET GLOBAL tikv_gc_concurrency = 3;
```

## Does `SHOW PROCESSLIST` display the system process ID?
Expand Down
4 changes: 2 additions & 2 deletions faq/tidb-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,12 +124,12 @@ The accessed Region is not available. A Raft Group is not available, with possib

#### ERROR 9006 (HY000): GC life time is shorter than transaction duration

The interval of `GC Life Time` is too short. The data that should have been read by long transactions might be deleted. You can add `GC Life Time` using the following command:
The interval of `GC Life Time` is too short. The data that should have been read by long transactions might be deleted. You can adjust [`tikv_gc_life_time`](/system-variables.md#tikv_gc_life_time) using the following command:

{{< copyable "sql" >}}

```sql
update mysql.tidb set variable_value='30m' where variable_name='tikv_gc_life_time';
SET GLOBAL tikv_gc_life_time = '30m';
```

> **Note:**
Expand Down
2 changes: 1 addition & 1 deletion garbage-collection-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,4 @@ In this step, TiDB only needs to send the safe point to PD, and then the whole r

> **Note:**
>
> In TiDB v2.1 or earlier versions, the Do GC step is implemented by TiDB sending requests to each Region. In v3.0 or later versions, you can modify the `tikv_gc_mode` to use the previous GC mechanism. For more details, refer to [GC Configuration](/garbage-collection-configuration.md#tikv_gc_mode).
> In TiDB v2.1 or earlier versions, the Do GC step is implemented by TiDB sending requests to each Region. In v3.0 or later versions, you can modify the [`tikv_gc_mode`](/system-variables.md#tikv_gc_mode) to use the previous GC mechanism.
10 changes: 5 additions & 5 deletions read-historical-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,19 @@ TiDB implements a feature to read history data using the standard SQL interface

## How TiDB reads data from history versions

The `tidb_snapshot` system variable is introduced to support reading history data. About the `tidb_snapshot` variable:
The [`tidb_snapshot`](/system-variables.md#tidb_snapshot) system variable is introduced to support reading history data. About the `tidb_snapshot` variable:

- The variable is valid in the `Session` scope.
- Its value can be modified using the `Set` statement.
- The variable is valid in the `SESSION` scope.
- Its value can be modified using the `SET` statement.
- The data type for the variable is text.
- The variable accepts TSO (Timestamp Oracle) and datetime. TSO is a globally unique time service, which is obtained from PD. The acceptable datetime format is "2016-10-08 16:45:26.999". Generally, the datetime can be set using second precision, for example "2016-10-08 16:45:26".
- When the variable is set, TiDB creates a Snapshot using its value as the timestamp, just for the data structure and there is no any overhead. After that, all the `Select` operations will read data from this Snapshot.
- When the variable is set, TiDB creates a Snapshot using its value as the timestamp, just for the data structure and there is no any overhead. After that, all the `SELECT` operations will read data from this Snapshot.

> **Note:**
>
> Because the timestamp in TiDB transactions is allocated by Placement Driver (PD), the version of the stored data is also marked based on the timestamp allocated by PD. When a Snapshot is created, the version number is based on the value of the `tidb_snapshot` variable. If there is a large difference between the local time of the TiDB server and the PD server, use the time of the PD server.
After reading data from history versions, you can read data from the latest version by ending the current Session or using the `Set` statement to set the value of the `tidb_snapshot` variable to "" (empty string).
After reading data from history versions, you can read data from the latest version by ending the current Session or using the `SET` statement to set the value of the `tidb_snapshot` variable to "" (empty string).

## How TiDB manages the data versions

Expand Down
8 changes: 4 additions & 4 deletions system-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -870,8 +870,8 @@ SET tidb_slow_log_threshold = 200;
- Scope: GLOBAL
- Default value: `DISTRIBUTED`
- Specifies the GC mode. Possible values are:
- `DISTRIBUTED`: Distributed GC mode. In the [Do GC](/garbage-collection-overview.md#do-gc) step, the GC leader on the TiDB side uploads the safe point to PD. Each TiKV node obtains the safe point respectively and performs GC on all leader Regions on the current node. This mode is is supported from TiDB 3.0.
- `CENTRAL`: Central GC mode. In the [Do GC](/garbage-collection-overview.md#do-gc) step, the GC leader sends GC requests to all Regions. This mode is adopted by TiDB 2.1 or earlier versions.
- `DISTRIBUTED`: Distributed GC mode. In the [Do GC](/garbage-collection-overview.md#do-gc) step, the GC leader on the TiDB side uploads the safe point to PD. Each TiKV node obtains the safe point respectively and performs GC on all leader Regions on the current node. This mode is supported from TiDB 3.0.
- `CENTRAL`: Central GC mode. In the [Do GC](/garbage-collection-overview.md#do-gc) step, the GC leader sends GC requests to all Regions. This mode is adopted by TiDB 2.1 or earlier versions. Starting from TiDB 5.0, this mode is not supported. Clusters set to this mode automatically switch to the `DISTRIBUTED` mode.
### tikv_gc_run_interval
Expand All @@ -888,8 +888,8 @@ SET tidb_slow_log_threshold = 200;
- Scope: GLOBAL
- Default value: `LEGACY`
- Possible values:
- `LEGACY`: Uses the old way of scanning, that is, disable Green GC.
- `PHYSICAL`: Uses the physical scanning method, that is, enable Green GC.
- `LEGACY`: Uses the old way of scanning, that is, disable Green GC.
- `PHYSICAL`: Uses the physical scanning method, that is, enable Green GC.
- This parameter specifies the way of scanning locks in the Resolve Locks step of GC, that is, whether to enable Green GC (experimental feature) or not. In the Resolve Locks step of GC, TiKV needs to scan all locks in the cluster. With Green GC disabled, TiDB scans locks by Regions. Green GC provides the "physical scanning" feature, which means that each TiKV node can bypass the Raft layer to directly scan data. This feature can effectively mitigate the impact of GC wakening up all Regions when the [Hibernate Region](/tikv-configuration-file.md#raftstorehibernate-regions-experimental) feature is enabled, thus improving the execution speed in the Resolve Locks step.
### time_zone
Expand Down

0 comments on commit 3b87cac

Please sign in to comment.