perf(prune): Only analyze when rebuilding #5186
Conversation
| // Analyze the new tables | ||
| let tables = prunable_tables.iter().map(|(table, _)| *table).collect(); | ||
| self.analyze_tables(conn, reporter, tables, cancel)?; | ||
|
|
There was a problem hiding this comment.
With default settings, Delete will be used if we think we'll delete between 5% and 50% of the table, where the latter is a lot and statistics from before deletion will likely be junk. It would be better to analyze the tables in the Delete branch for which catalog::needs_autoanalyze says that an analyze is needed. That would basically cut out waiting for the autoanalyze daemon to get around to analyzing these tables.
leoyvens
force-pushed
the
leo/overanalyzed
branch
from
0177d43
to
d21ad66
Compare
|
@lutter thanks for reviewing, I've tried another approach on the latest commit, given I've learned about the very convenient |
I don't think we have a minimum version documented, but not worrying about EOL versions seems reasonable to me. |
When pruning by delete, it's overzealous to analyze. When combined with auto-prune on large tables, I've seen shards struggle with one long analyze after the other. It can also conflict with the lock for autovacuum and result in a long running transaction waiting for the lock.
leoyvens
force-pushed
the
leo/overanalyzed
branch
from
d21ad66
to
b93b0d2
Compare
leoyvens
force-pushed
the
leo/overanalyzed
branch
from
b93b0d2
to
52d9c6a
Compare
When pruning by delete, it's overzealous to analyze. When combined with auto-prune on large tables, I've seen shards struggle with one long analyze after the other.
It can also conflict with the lock for autovacuum and result in a long running transaction waiting for the lock.