Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dumpling v6.1 still modify GC lift time by update the mysql.tidb table #37143

Closed
Mini256 opened this issue Aug 16, 2022 · 6 comments · May be fixed by #37189
Closed

Dumpling v6.1 still modify GC lift time by update the mysql.tidb table #37143

Mini256 opened this issue Aug 16, 2022 · 6 comments · May be fixed by #37189
Assignees
Labels
affects-6.1 component/dumpling This is related to Dumpling of TiDB. severity/moderate type/bug The issue is confirmed as a bug.

Comments

@Mini256
Copy link
Member

Mini256 commented Aug 16, 2022

Bug Report

1. Minimal reproduce step (Required)

tiup dumpling --host <host> --port 4000 --user root --password '<password>' --filetype sql --filter "test.<table_name>" --where="<condition>" --output ./

2. What did you expect to see? (Required)

Maybe we should use the global variable tidb_gc_life_time to do it.

3. What did you see instead (Required)

[2022/08/16 13:45:48.267 +08:00] [INFO] [versions.go:55] ["Welcome to dumpling"] ["Release Version"=v6.1.0] ["Git Commit Hash"=1a89decdb192cbdce6a7b0020d71128bc964d30f] ["Git Branch"=heads/refs/tags/v6.1.0] ["Build timestamp"="2022-06-05 05:09:35"] ["Go Version"="go version go1.18.2 darwin/arm64"]
[2022/08/16 13:45:49.317 +08:00] [INFO] [version.go:362] ["detect server version"] [type=TiDB] [version=6.1.0-20220729]
[2022/08/16 13:45:49.797 +08:00] [WARN] [dump.go:1408] ["If the amount of data to dump is large, criteria: (data more than 60GB or dumped time more than 10 minutes)\nyou'd better adjust the tikv_gc_life_time to avoid export failure due to TiDB GC during the dump process.\nBefore dumping: run sql `update mysql.tidb set VARIABLE_VALUE = '720h' where VARIABLE_NAME = 'tikv_gc_life_time';` in tidb.\nAfter dumping: run sql `update mysql.tidb set VARIABLE_VALUE = '10m' where VARIABLE_NAME = 'tikv_gc_life_time';` in tidb.\n"]
[2022/08/16 13:45:51.336 +08:00] [INFO] [dump.go:1463] ["cannot check whether TiDB has TiKV, will apply tidb_snapshot by default. This won't affect dump process"] [error="sql: SELECT COUNT(1) as c FROM MYSQL.TiDB WHERE VARIABLE_NAME='tikv_gc_safe_point': Error 1142: SELECT command denied to user 'root'@'%' for table 'tidb'"]
mysql> use mysql;
You are now connected to database "mysql" as user "root"
Time: 0.224s
mysql> SHOW tables LIKE 'tidb';
+-----------------+
| Tables_in_mysql |
+-----------------+
+-----------------+
0 rows in set
Time: 0.233s

4. What is your TiDB version? (Required)

Release Version: v6.1.0-20220729 (on TiDB Cloud)

@Mini256 Mini256 added the type/bug The issue is confirmed as a bug. label Aug 16, 2022
@Mini256
Copy link
Member Author

Mini256 commented Aug 16, 2022

/component dumpling

@ti-chi-bot ti-chi-bot added the component/dumpling This is related to Dumpling of TiDB. label Aug 16, 2022
@jebter
Copy link

jebter commented Aug 17, 2022

Maybe tidb_gc_enable is a better way

@morgo
Copy link
Contributor

morgo commented Aug 17, 2022

Using the mysql.tidb table directly is currently safe, but it's not recommended (and doesn't work in TiDB Cloud where access to this table is prohibited). Since #21988 get/set to the system variable proxies just writing to this table, but in future it should be a first class system variable.

On my system, I have 30ms p80s for SHOW commands :( It is because of the GC variables needing to be read from mysql.tidb row-at-a-time. The SHOW commands are generated by client libraries, it's not even something the applications do.

@lichunzhu
Copy link
Contributor

It's just hint that dumpling reminds users to set a bigger gc life time. Actually dumpling won't do this sql.
I think this can't be regarded as a bug. A PR is released to fix this problem.

@Mini256
Copy link
Member Author

Mini256 commented Aug 18, 2022

It's just hint that dumpling reminds users to set a bigger gc life time.

Got it, but, judging from the log, dumpling will send the following SQL to TiDB to check whether TiDB uses TiKV. It seems that this does not work on TiDB cloud.

error="sql: SELECT COUNT(1) as c FROM MYSQL.TiDB WHERE VARIABLE_NAME='tikv_gc_safe_point': Error 1142: SELECT command denied to user 'root'@'%' for table 'tidb'"

@lichunzhu
Copy link
Contributor

lichunzhu commented Aug 18, 2022

It's just hint that dumpling reminds users to set a bigger gc life time.

Got it, but, judging from the log, dumpling will send the following SQL to TiDB to check whether TiDB uses TiKV. It seems that this does not work on TiDB cloud.

error="sql: SELECT COUNT(1) as c FROM MYSQL.TiDB WHERE VARIABLE_NAME='tikv_gc_safe_point': Error 1142: SELECT command denied to user 'root'@'%' for table 'tidb'"

Dumpling v6.1.0 is using select @@tidb_config at first. It sames that it fails too. But it still doesn't matter as the log says: "cannot check whether TiDB has TiKV, will apply tidb_snapshot by default. This won't affect dump process".

see https://github.com/pingcap/tidb/blob/v6.1.0/dumpling/export/sql.go#L793

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.1 component/dumpling This is related to Dumpling of TiDB. severity/moderate type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants