Skip to content

Latest commit

 

History

History
90 lines (60 loc) · 5.34 KB

ecosystem-tool-user-guide.md

File metadata and controls

90 lines (60 loc) · 5.34 KB
Raw
title aliases
TiDB Ecosystem Tools Overview
/docs/dev/ecosystem-tool-user-guide/
/docs/dev/reference/tools/user-guide/
/docs/dev/how-to/migrate/from-mysql/
/docs/dev/how-to/migrate/incrementally-from-mysql/
/docs/dev/how-to/migrate/overview/

TiDB Ecosystem Tools Overview

This document introduces the functionalities of TiDB ecosystem tools and their relationship.

Full data export

Dumpling is a tool for the logical full data export from MySQL or TiDB.

The following are the basics of Dumpling:

  • Input: MySQL/TiDB cluster
  • Output: SQL/CSV file
  • Supported TiDB versions: all versions
  • Kubernetes support: No

Note:

PingCAP previously maintained a fork of the mydumper project with enhancements specific to TiDB. This fork has since been replaced by Dumpling, which has been rewritten in Go, and supports more optimizations that are specific to TiDB. It is strongly recommended that you use Dumpling instead of mydumper.

Full data import

TiDB Lightning (Lightning) is a tool used for the full import of large amounts of data into a TiDB cluster. Currently, TiDB Lightning supports reading SQL dump exported via Dumpling or CSV data source.

TiDB Lightning supports three modes:

  • local: TiDB Lightning parses data into ordered key-value pairs and directly imports them into TiKV. This mode is usually for importing a large amount of data (at the TB level) to a new cluster. During the import, the cluster cannot provide services.
  • importer: This mode is similar to the local mode. To use this mode, you need to deploy an additional component tikv-importer to help import key-value pairs. If the target cluster is in v4.0 or later versions, it is recommended to use the local mode.
  • tidb: This mode uses TiDB/MySQL as the backend, which is slower than the local mode and importer mode but can be performed online. It also supports importing data to MySQL.

The following are the basics of TiDB Lightning:

Note:

The Loader tool is no longer maintained. For scenarios related to Loader, it is recommended that you use the tidb mode of TiDB Lighting instead. For details, see TiDB Lightning TiDB backends.

Backup and restore

Backup & Restore (BR) is a command-line tool for distributed backup and restore of the TiDB cluster data. BR can effectively back up and restore TiDB clusters of huge data volume.

The following are the basics of BR:

Incremental data replication

TiDB Binlog is a tool that collects binlog for TiDB clusters and provides near real-time sync and backup. It can be used for incremental data replication between TiDB clusters, such as making a TiDB cluster the secondary cluster of the primary TiDB cluster.

The following are the basics of TiDB Binlog:

Data migration

TiDB Data Migration (DM) is an integrated data replication task management platform that supports the full data migration and the incremental data replication from MySQL/MariaDB to TiDB.

The following are the basics of DM:

  • Input: MySQL/MariaDB
  • Output: TiDB cluster
  • Supported TiDB versions: all versions
  • Kubernetes support: No, under development

If the data volume is below the TB level, it is recommended to migrate data from MySQL/MariaDB to TiDB directly using DM. The migration process includes the full data import and export and the incremental data replication.

If the data volume is at the TB level, take the following steps:

  1. Use Dumpling to export the full data from MySQL/MariaDB.
  2. Use TiDB Lightning to import the data exported in Step 1 to the TiDB cluster.
  3. Use DM to replicate the incremental data from MySQL/MariaDB to TiDB.

Note:

The Syncer tool is no longer maintained. For scenarios related to Syncer, it is recommended that you use DM's incremental task mode instead.