Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing multi level index cache #5451

Merged
merged 4 commits into from
Jul 12, 2023

Conversation

alanprot
Copy link
Member

@alanprot alanprot commented Jul 11, 2023

What this PR does:
Implementing multi-level caching for the index cache.

With the PR we will be able to configure multiples caching layers like for example "inMemory" as L1 cache and "memcache" as l2 cache.

The configuration will now accept a Comma separated ordered list of index cache backend type and cortex will use the cache in order (l1,l2,ln).

Currently we don't "backfill" the data from a Ln+1 to a Ln cache but this can be implemented in the future.

PS: This change should be a NoOp if only 1 cache is configured.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Alan Protasio <[email protected]>
@alanprot alanprot force-pushed the multi-level-indexcache branch 2 times, most recently from 771c27f to bb01d9e Compare July 12, 2023 16:29
@alanprot alanprot marked this pull request as ready for review July 12, 2023 16:30
pkg/storage/tsdb/index_cache.go Outdated Show resolved Hide resolved
pkg/storage/tsdb/index_cache.go Show resolved Hide resolved
pkg/storage/tsdb/index_cache.go Outdated Show resolved Hide resolved
Signed-off-by: Alan Protasio <[email protected]>
Copy link
Contributor

@harry671003 harry671003 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks Alan.

Signed-off-by: Alan Protasio <[email protected]>
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@yeya24 yeya24 merged commit 241fa91 into cortexproject:master Jul 12, 2023
14 checks passed
qinxx108 pushed a commit to qinxx108/cortex that referenced this pull request Jul 13, 2023
qinxx108 pushed a commit to qinxx108/cortex that referenced this pull request Jul 13, 2023
alvinlin123 pushed a commit that referenced this pull request Jul 13, 2023
* Implementing Bucket index sync status file (#5446)

* Implementing Bucket index sync status

Signed-off-by: Alan Protasio <[email protected]>

* fixing bug when returning from cache

Signed-off-by: Alan Protasio <[email protected]>

* Addressing some comments

Signed-off-by: Alan Protasio <[email protected]>

* Changelog

Signed-off-by: Alan Protasio <[email protected]>

---------

Signed-off-by: Alan Protasio <[email protected]>
Signed-off-by: Yijie Qin <[email protected]>

* Implementing multi level index cache (#5451)

Signed-off-by: Yijie Qin <[email protected]>

* remove the user from am state replication key

Signed-off-by: Yijie Qin <[email protected]>

* expose the key label

Signed-off-by: Yijie Qin <[email protected]>

* add Changelog

Signed-off-by: Yijie Qin <[email protected]>

* fix comments

Signed-off-by: Yijie Qin <[email protected]>

* change to use type

Signed-off-by: Yijie Qin <[email protected]>

* address comment

Signed-off-by: Yijie Qin <[email protected]>

* fix comment

Signed-off-by: Yijie Qin <[email protected]>

* use type instead of key

Signed-off-by: Yijie Qin <[email protected]>

---------

Signed-off-by: Alan Protasio <[email protected]>
Signed-off-by: Yijie Qin <[email protected]>
Co-authored-by: Alan Protasio <[email protected]>
@alanprot alanprot deleted the multi-level-indexcache branch August 16, 2023 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants