Don't load global state when only restoring indices #28934
Labels
Comments
ywelsch
added
discuss
:Distributed/Snapshot/Restore
|
Seems logical to me. @tlrx , any opinions? |
It makes sense to me. I think that we can have the same issue when deleting a snapshot but I'm not sure we can avoid loading the global state in this case. I'll take a look. |
hub-cap
added
:Distributed/Snapshot/Restore
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this issue
Mar 25, 2018
Restoring a snapshot, or getting the status of finished snapshots, currently always load the global state metadata file from the repository even if it not required. This slows down the restore process (or listing statuses process) and can also be an issue if the global state cannot be deserialized (because it has unknown customs for example). This commit splits the Repository.getSnapshotMetadata() method into two distincts methods: getGlobalMetadata() and getIndexMetadata() that are now called on purpose. Closes elastic#28934
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this issue
Mar 28, 2018
When deleting a snapshot, it is not necessary to load and to parse the global metadata of the snapshot to delete. Now indices are stored in the snapshot metadata file, we have all the information to resolve the shards files to delete. This commit removes the readSnapshotMetaData() method that was used to load both global and index metadata files. Test coverage should be enough as SharedClusterSnapshotRestoreIT already contains several deletion tests. Related to elastic#28934
tlrx
added a commit
that referenced
this issue
Mar 29, 2018
When deleting a snapshot, it is not necessary to load and to parse the global metadata of the snapshot to delete. Now indices are stored in the snapshot metadata file, we have all the information to resolve the shards files to delete. This commit removes the readSnapshotMetaData() method that was used to load both global and index metadata files. Test coverage should be enough as SharedClusterSnapshotRestoreIT already contains several deletion tests. Related to #28934
tlrx
added a commit
that referenced
this issue
May 3, 2018
When deleting a snapshot, it is not necessary to load and to parse the global metadata of the snapshot to delete. Now indices are stored in the snapshot metadata file, we have all the information to resolve the shards files to delete. This commit removes the readSnapshotMetaData() method that was used to load both global and index metadata files. Test coverage should be enough as SharedClusterSnapshotRestoreIT already contains several deletion tests. Related to #28934
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Restoring from a snapshot currently also always loads the global state (templates, customs, ...) even though it's not always used by the restore process, for example when
include_global_stateis set tofalsein the restore request. In case there are version incompatibilities in the global state file, or corruption issues, this prevents indices from being restored. We could instead simply not load the global state when it's not required for the restore process./cc: @elastic/es-distributed
The text was updated successfully, but these errors were encountered: