Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log-backup: failed to restore index using PiTR #37111

Closed
joccau opened this issue Aug 15, 2022 · 1 comment · Fixed by #37100
Closed

log-backup: failed to restore index using PiTR #37111

joccau opened this issue Aug 15, 2022 · 1 comment · Fixed by #37100
Assignees
Labels
affects-6.2 component/br This issue is related to BR of TiDB. found/automation Found by automation tests severity/critical type/bug The issue is confirmed as a bug.

Comments

@joccau
Copy link
Member

joccau commented Aug 15, 2022

Bug Report

Please answer these questions before submitting your issue. Thanks!

The detail issue on https://pingcap.feishu.cn/wiki/wikcnPeLF5XEf1Ls04e6vpWdIMd

  1. TiDB version
    6.2.0-alpha TiDB cluster and br

  2. Reproduce steps.

  • Start log backup
  • Do full backup
  • Start workload with 1000 tables for 10 min
    sysbench --db-driver=mysql --mysql-host=upstream-tidb.brie-acceptance-pitr-robust-tps-1083099-1-452 --mysql-port=4000 --mysql-user=root --mysql-db=sbtest1000 --tables=1000 --table-size=10000 --create_secondary=off --time=60 --threads=20 oltp_insert prepare sysbench --db-driver=mysql --mysql-host=upstream-tidb.brie-acceptance-pitr-robust-tps-1083099-1-452 --mysql-port=4000 --mysql-user=root --mysql-db=sbtest1000 --tables=1000 --table-size=10000 --create_secondary=off --time=600 --threads=20 oltp_insert run
  • Wait checkpoint move forward
    `
    ● Total 1 Tasks.

#1 <
name: task1
status: ● NORMAL
start: 2022-08-13 02:28:36.516 +0000
end: 2090-11-18 14:07:45.624 +0000
storage: s3://tmp/br-log-backup84ed324a-894c-4f13-84fa-44675095787d/Log
speed(est.): 0.00 ops/s
checkpoint[global]: 2022-08-13 02:43:07.216 +0000; gap=4m59s
[2022/08/13 10:48:06.135 +08:00] [INFO] [br.go:518] ["Current log backup checkpoint: 2022-08-13 02:43:07 +0000 UTC, expect it is larger than: 2022-08-13 02:41:05.816 +0000 UTC"]
`

  • Do a PiTR restore, and check data consistency
    /br restore point "--storage" "s3://tmp/br-log-backup84ed324a-894c-4f13-84fa-44675095787d/Log" "--pd" "http://downstream-pd.brie-acceptance-pitr-robust-tps-1083099-1-452:2379" "--full-backup-storage" "s3://tmp/br-log-backup84ed324a-894c-4f13-84fa-44675095787d/Full"

You will find the index on table in down steam cluster is disappeared.

@joccau joccau added the type/bug The issue is confirmed as a bug. label Aug 15, 2022
@joccau
Copy link
Member Author

joccau commented Aug 15, 2022

/assign

@joccau joccau added component/br This issue is related to BR of TiDB. severity/critical labels Aug 15, 2022
@ti-chi-bot ti-chi-bot added may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 labels Aug 15, 2022
@joccau joccau added affects-6.2 and removed may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 labels Aug 15, 2022
joccau added a commit to joccau/tidb that referenced this issue Aug 16, 2022
@fubinzh fubinzh added the found/automation Found by automation tests label Aug 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.2 component/br This issue is related to BR of TiDB. found/automation Found by automation tests severity/critical type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants