Support the ignore missing key read option in recurse_index_keys #1844
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is required for delayed deletes to work correctly.
poodlewars
requested review from
alexowens90 and
willdealtry
as code owners
poodlewars
commented
Sep 20, 2024
| for (auto &&key : sub_keys) { | ||
| res.emplace(std::move(key)); | ||
| } | ||
| } else if (index_key.type() == KeyType::TABLE_INDEX) { |
There was a problem hiding this comment.
I considered pulling out the different behaviours based on type to functions as this is getting quite messy, but the increased indirection made things harder to follow rather than easier.
| if (opts.ignores_missing_key_) { | ||
| log::version().info("Missing key while recursing index key {}", e.keys()); | ||
| } else { | ||
| throw storage::KeyNotFoundException(std::move(e.keys())); |
There was a problem hiding this comment.
nit
Suggested change
| throw storage::KeyNotFoundException(std::move(e.keys())); | |
| throw; |
will rethrow the current excepion
vasil-pashov
approved these changes
Sep 30, 2024
willdealtry
approved these changes
Sep 30, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is required for delayed deletes to work correctly - https://github.com/man-group/arcticdb-enterprise/pull/167 includes an end to end failing test to motivate this.
The issue is that if we have a snapshot being modified over time, replication might sync over its current state from the source storage before the index keys inside it. So delayed deletes needs to be tolerant to the fact that some of the keys it is asked to recurse may not actually exist.
I also noticed that
recurse_index_keysrequires all the index keys it is run over to be for the same symbol. I'm going to check that we are in fact doing that, and add some debug checks, as a follow up.