Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

meta: delay deletion of compacted slices #1790

Merged
merged 22 commits into from
Apr 21, 2022
Merged

meta: delay deletion of compacted slices #1790

merged 22 commits into from
Apr 21, 2022

Conversation

SandyXSD
Copy link
Contributor

@SandyXSD SandyXSD commented Apr 14, 2022

close #1521

TODO:

  • Dump & Load: need a new struct DumpedDelSlice
    • better not load the delSlices, because the whole sliceRefs is rebuild when loading entries, and it's dangerous to further decrease any of the refs. Instead we can just run gc --delete command after loading to clean up the leaked slices.
  • Unit tests
  • SQL: need a new table: delslices{chunkid, ts, blob}
  • TKV: Lcccccccctttttttt -> blob

@davies davies self-requested a review April 19, 2022 09:58
@codecov-commenter
Copy link

codecov-commenter commented Apr 20, 2022

Codecov Report

Merging #1790 (7c81854) into main (c76cf44) will decrease coverage by 0.21%.
The diff coverage is 61.51%.

❗ Current head 7c81854 differs from pull request most recent head 8174d12. Consider uploading reports for the commit 8174d12 to get more accurate results

@@            Coverage Diff             @@
##             main    #1790      +/-   ##
==========================================
- Coverage   61.90%   61.69%   -0.22%     
==========================================
  Files         135      135              
  Lines       23011    23246     +235     
==========================================
+ Hits        14246    14342      +96     
- Misses       7171     7272     +101     
- Partials     1594     1632      +38     
Impacted Files Coverage Δ
pkg/meta/base.go 69.82% <38.46%> (-1.87%) ⬇️
pkg/meta/redis.go 72.81% <55.85%> (-0.43%) ⬇️
pkg/meta/sql.go 66.50% <68.49%> (-0.55%) ⬇️
pkg/meta/tkv.go 76.14% <71.21%> (-0.32%) ⬇️
pkg/sync/sync.go 45.17% <100.00%> (+0.19%) ⬆️
pkg/fuse/context.go 56.75% <0.00%> (-10.82%) ⬇️
pkg/vfs/reader.go 80.29% <0.00%> (-4.38%) ⬇️
pkg/chunk/cached_store.go 76.05% <0.00%> (-2.20%) ⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5d77807...8174d12. Read the comment docs.

pkg/meta/base.go Outdated Show resolved Hide resolved
pkg/meta/redis_test.go Outdated Show resolved Hide resolved
@SandyXSD SandyXSD requested a review from davies April 21, 2022 13:03
@davies davies merged commit 1eb0d3d into main Apr 21, 2022
@davies davies deleted the delay-delete-slices branch April 21, 2022 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Delay deletion of old slices after compaction if trash is enabled
3 participants