Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support to export historical statistics #18745

Open
qw4990 opened this issue Jul 23, 2020 · 3 comments · Fixed by #31032
Open

Support to export historical statistics #18745

qw4990 opened this issue Jul 23, 2020 · 3 comments · Fixed by #31032
Assignees
Labels
component/statistics feature/accepted This feature request is accepted by product managers sig/planner SIG: Planner type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@qw4990
Copy link
Contributor

qw4990 commented Jul 23, 2020

Feature Request

Is your feature request related to a problem? Please describe:

There are some cases that when we(developer) get the wrong execution plan, statistics have been updated by auto-analyze or users without intention, which makes it harder to reproduce the plan and locate the reason.

Describe the feature you'd like:

Support to export historical statistics.
There is already an API that can do this, but it has a strict limitation that the specified time cannot be earlier tikv_gc_run_interval minutes than the current time. (The default value of tikv_gc_run_interval is 10 minutes)
This limitation makes this API hard to be used in realistic cases.

It's better to design a new way to implement this feature.

Teachability, Documentation, Adoption, Migration Strategy:

Export statistics in TiDB

@qw4990 qw4990 added component/statistics type/feature-request Categorizes issue or PR as related to a new feature. labels Jul 23, 2020
@qw4990 qw4990 self-assigned this Jul 23, 2020
@qw4990
Copy link
Contributor Author

qw4990 commented Jul 23, 2020

We can use this feature(#18117) to remove the limitation.

@qw4990 qw4990 closed this as completed Jul 23, 2020
@zz-jason zz-jason added the feature/accepted This feature request is accepted by product managers label Jul 29, 2020
@SunRunAway
Copy link
Contributor

SunRunAway commented Sep 1, 2020

I reopened this issue because extra works will be needed after #18117 is finished.

  1. an interface to let users dump historical statistics according to a timestamp
  2. a default GC time for the statistics tables.

@SunRunAway SunRunAway reopened this Sep 1, 2020
@zz-jason zz-jason added the feature/reviewing This feature request is reviewing by product managers label Sep 1, 2020
@zz-jason zz-jason removed the feature/reviewing This feature request is reviewing by product managers label Sep 3, 2020
@qw4990
Copy link
Contributor Author

qw4990 commented Dec 8, 2021

The design doc(Chinese version) designed by @An-DJ and the work items:

  • introduce a new global variable as the switch to control the behavior
  • support to record historical meta stats when updating mysql.stats_meta_history
  • support to record historical main stats when updating histogram, topN and etc.
  • GC stale historical stats data
  • update the original HTTP API to support to dump historical stats from the two new tables

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/statistics feature/accepted This feature request is accepted by product managers sig/planner SIG: Planner type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants