Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve admin dashboard summary display to prevent timeout #17506

Closed
wxiaoguang opened this issue Oct 31, 2021 · 3 comments · Fixed by #19775
Closed

Improve admin dashboard summary display to prevent timeout #17506

wxiaoguang opened this issue Oct 31, 2021 · 3 comments · Fixed by #19775
Labels
type/enhancement An improvement of existing functionality

Comments

@wxiaoguang
Copy link
Contributor

wxiaoguang commented Oct 31, 2021

Description

The Admin -> Dashboard -> Summary shows many counting numbers. Some of them aren't useful to admins. Even worse, some counting operations on the large tables like action will make the page timeout. For example:

I suggest to improve the summary information, we shouldn't count so many models. Each number displayed here should be useful to admins.

@wxiaoguang wxiaoguang changed the title Admin The counting operations on the admin dashboard summary page may timeout Oct 31, 2021
@wxiaoguang wxiaoguang changed the title The counting operations on the admin dashboard summary page may timeout Improve admin dashboard summary display to prevent timeout Oct 31, 2021
@lunny lunny added the type/enhancement An improvement of existing functionality label Nov 1, 2021
@zeripath
Copy link
Contributor

zeripath commented Apr 23, 2022

Instead of these COUNT(*) calls I think we need to use something like:

SELECT table_rows FROM information_schema.tables WHERE tables.table_name = ? AND tables.table_schema = ?;

with the first parameter as the table name and the second param is the db name.


Of course table_rows is a MySQL extension.

Postgres needs:

SELECT reltuples AS estimate FROM pg_class WHERE relname = 'table_name';

SQLite it appears that COUNT is just the fastest available.


MSSQL

sp_spaceused 'Table_name`

zeripath added a commit to zeripath/gitea that referenced this issue Apr 23, 2022
Currently whenever the prometheus metrics endpoint or `/admin` endpoint are viewed
the statistics are recalculated immediately - using COUNT rather than a less expensive
method.

This PR provides a mechanism to cache these statistics, avoids generating all of the
metrics on the admin page and provides an estimation method for the plain table counts.

Fix go-gitea#17506

Signed-off-by: Andrew Thornton <[email protected]>
@zeripath
Copy link
Contributor

zeripath commented May 21, 2022

I have to say I suspect that getting the count for the Action table might be a false report - the real problems are if you have:

[metrics]
ENABLED_ISSUE_BY_LABEL=true 
ENABLED_ISSUE_BY_REPOSITORY=true

Then all of that work occurs every time you look at the admin dashboard page.


Thus we have to cache these statistics or remove these from the GetStatistics call.

@wxiaoguang
Copy link
Contributor Author

wxiaoguang commented May 21, 2022

For the user (#17504), it's not a false report, after the user truncated their action table, then their system functions correctly.

For me, it's not a false report. Now what I am doing is truncating my action table regularly. (I know how to tune MySQL to bypass the problem, but the truncating just reminds me that there is still a problem).


ps: other statistics also have performance problem, too, so caching them and moving them to a dedicated page are all good.

@go-gitea go-gitea locked and limited conversation to collaborators May 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type/enhancement An improvement of existing functionality
Projects
None yet
3 participants