Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Timestamp Pinning in Remote Store] Scheduler to fetch pinned timestamp list from remote store #15062

Closed
sachinpkale opened this issue Aug 1, 2024 · 0 comments
Assignees
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Remote

Comments

@sachinpkale
Copy link
Member

sachinpkale commented Aug 1, 2024

Is your feature request related to a problem? Please describe

Given that remote store garbage collectors are aware of pinned timestamps, it becomes important to communicate any change to pinned timestamp to all the nodes in cluster within pre-defined SLA.

Describe the solution you'd like

  • To make sure each node gets the updated pinned timestamp details within X mins, in this solution, we run a scheduler on each node with period of X/2 mins (this is to make sure we run the scheduler at least twice).
  • This scheduler will fetch the list of pinned timestamps and store it along with timestamp of the successful fetch.
  • The timestamp of successful is required for garbage collector to identify if it is accessing stale pinned timestamp list and in such cases, garbage collector needs to be skipped.

Related component

Storage:Remote

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Remote
Projects
Status: ✅ Done
Development

No branches or pull requests

1 participant