Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

lightning: fix several disk-quota-related bugs #815

Closed
wants to merge 5 commits into from

Conversation

kennytm
Copy link
Collaborator

@kennytm kennytm commented Mar 5, 2021

What problem does this PR solve?

  1. Fix Lightning Disk Quota is not fully respected after restart due to non-deterministic engine selection #812
  2. Fix the bug where the checkpoints in local back-end are not advanced even after disk quota force-flushed all engines.

What is changed and how it works?

  1. For Lightning Disk Quota is not fully respected after restart due to non-deterministic engine selection #812, it is caused by iterating cp.Engines in restoreEngines(), which is a hash map. We read the map into a sorted array and iterate that instead, so engines are always restored in deterministic order.
  2. For the checkpoint issue, we now update the checkpoint based on the Local Writer's synchronization status instead of the disk quota status (the deliverLoop cannot see diskQuotaStateImporting during diskQuotaLock.RLock()). This assumes Pebble's Ingest() will persist all content to disk.

Check List

Tests

  • Integration test (TODO)
  • Manual test (add detailed scripts or steps below)
    • Testing its effect in the 1T GA test.

Code changes

Side effects

Related changes

  • Need to cherry-pick to the release branch

Release Note

  • No release note.

@kennytm kennytm changed the title lightning: Fix sevreal disk-quota-related bugs lightning: fix several disk-quota-related bugs Mar 5, 2021
@kennytm kennytm force-pushed the fix-disk-quota-ga-bugs branch 4 times, most recently from 30def51 to 3fcc3a6 Compare March 5, 2021 19:57
@kennytm kennytm force-pushed the fix-disk-quota-ga-bugs branch 12 times, most recently from 8b2aa37 to 120e149 Compare March 6, 2021 20:27
@kennytm
Copy link
Collaborator Author

kennytm commented Mar 11, 2021

Separated into #761, #845, #846, #847, closing.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Lightning Disk Quota is not fully respected after restart due to non-deterministic engine selection
2 participants