Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance the memory usage of mark cache and minmax index cache #8050

Closed
hongyunyan opened this issue Sep 4, 2023 · 1 comment
Closed

Enhance the memory usage of mark cache and minmax index cache #8050

hongyunyan opened this issue Sep 4, 2023 · 1 comment

Comments

@hongyunyan
Copy link
Contributor

Enhancement

Currently, in minmax_index_cache and mark_cache we use the long file name as the key, which occupy much memory, especially when the memory cost of value is small(with a little pack number).

Besides, we statistic memory overhead of mark cache size, but only counted about the memory cost of MarksInCompressedFile, but not consider the memory cost of keys, and the structual cost of LRUCache, which cause a misleading of memory usage.

Thus, we should optimize the memory usage of these caches, and make a better memory statistical for these memory usage of caches.

@JaySon-Huang
Copy link
Contributor

JaySon-Huang commented Sep 7, 2023

Change the type from enhancement to bug because it leads to much more memory consumption than expected under some scenario

@JaySon-Huang JaySon-Huang added type/bug The issue is confirmed as a bug. severity/moderate affects-6.5 affects-7.1 and removed type/enhancement The issue or PR belongs to an enhancement. labels Sep 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants