Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Migrations: Improve Disabling and Restoration of ElasticSearch Indexing Settings #4187

Merged
merged 1 commit into from
Jun 28, 2024

Conversation

adrians5j
Copy link
Member

@adrians5j adrians5j commented Jun 27, 2024

Changes

With this PR, we're improving the disablement and restoration of ES indexing settings. Having these improvements would also be useful for previous data migrations, but for now, we won't be porting them to those. We'll start thinking about it when an actual need arises.

First of all, when disabling indexing on an index, we no longer set the number_of_replicas to zero. From this doc:

This will temporarily put your index at risk since the loss of any shard will cause data loss, but at the same time indexing will be faster since ...

So, because there is risk involved, we decided to simply not fiddle with this setting.

Furthermore, we've addressed an index restoration issue. Prior to this PR, when doing a restoration, previously retrieved ES index would be used. Which is OK, because this way we ensure the original index settings are applied back to the inedx.

But, if a user stopped the data migration manually, and then rerun it, then the restoration step would incorrectly again set "disable indexing" settings. This is because it would use the index settings that were retrieved at the beginning of the new run, which, because the previous run has been manually stopped, still contain refresh_interval: -1. Ultimately, this setting would be applied when restoring index settings, effectively leaving the "indexing: disabled" setting still active.

How Has This Been Tested?

Manually. Relying on existing Jest tests.

Documentation

Changelog.

@adrians5j adrians5j changed the base branch from next to 5.39.9 June 27, 2024 20:31
@adrians5j adrians5j changed the title fix: improve disabling and restoration of ES indexing settings Data Migrations: Improve Disabling and Restoration of ElasticSearch Indexing Settings Jun 27, 2024
@adrians5j adrians5j added this to the 5.39.9 milestone Jun 27, 2024
@adrians5j adrians5j merged commit d65c7e6 into 5.39.9 Jun 28, 2024
94 checks passed
@adrians5j adrians5j deleted the adrian/refresh-interval-fallback branch July 5, 2024 07:00
@adrians5j adrians5j restored the adrian/refresh-interval-fallback branch August 8, 2024 12:46
@adrians5j adrians5j deleted the adrian/refresh-interval-fallback branch August 22, 2024 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant