Use data stream for ILM history #64521
Conversation
|
@elasticmachine update branch |
probakowski
added
:Data Management/ILM+SLM
|
Pinging @elastic/es-core-features (:Core/Features/ILM+SLM) |
|
Note: this supersedes #60280 with basically cleaned up version of the same code |
| @@ -646,7 +646,8 @@ private void wipeCluster() throws Exception { | |||
| protected static void wipeAllIndices() throws IOException { | |||
| boolean includeHidden = minimumNodeVersion().onOrAfter(Version.V_7_7_0); | |||
| try { | |||
| final Request deleteRequest = new Request("DELETE", "*"); | |||
| //remove all indices except ilm history which can pop up after deleting all data streams but shouldn't interfere | |||
| final Request deleteRequest = new Request("DELETE", "*,-.ds-ilm-history-*"); | |||
There was a problem hiding this comment.
This looks fishy but it's similar situation as we had before - ILM history index could pop up just after deleting all indices and it didn't do any harm
| if (ilmHistoryEnabled == false) { | ||
| throw new ElasticsearchException("can not index ILM history items when ILM history is disabled"); | ||
| } |
There was a problem hiding this comment.
I don't think we need this check here as well.
Is this meant to catch the case when the ilm history was disabled between the time putAsync was called and the bulk was actually executed? If so, I think it'd be fine to allow it to go through as opposed to failing (and the next putAsync call will not record any history items anymore)
If we do decide to keep it I believe the exception should report the items it had not indexed.
What do you think?
There was a problem hiding this comment.
Since the indices.lifecycle.history_index_enabled setting is not dynamic, I don't think we need this check (since putAsync will always punt)
There was a problem hiding this comment.
You are right, I've removed it
| if (ilmHistoryEnabled == false) { | ||
| throw new ElasticsearchException("can not index ILM history items when ILM history is disabled"); | ||
| } |
There was a problem hiding this comment.
Since the indices.lifecycle.history_index_enabled setting is not dynamic, I don't think we need this check (since putAsync will always punt)
|
@elasticmachine update branch |
This change moves ILM history from using normal index and legacy template to data streams and composable templates. It also unmutes several ITs to check if using data streams will resolve some race conditions there. # Conflicts: # x-pack/plugin/ilm/src/internalClusterTest/java/org/elasticsearch/xpack/ilm/ILMHistoryTests.java # x-pack/plugin/ilm/src/main/java/org/elasticsearch/xpack/ilm/history/ILMHistoryStore.java
* Use data stream for ILM history (#64521) This change moves ILM history from using normal index and legacy template to data streams and composable templates. It also unmutes several ITs to check if using data streams will resolve some race conditions there. # Conflicts: # x-pack/plugin/ilm/src/internalClusterTest/java/org/elasticsearch/xpack/ilm/ILMHistoryTests.java # x-pack/plugin/ilm/src/main/java/org/elasticsearch/xpack/ilm/history/ILMHistoryStore.java * fix unused imports
This change moves ILM history from using normal index and legacy template to data streams and composable templates.
It also unmutes several ITs to check if using data streams will resolve some race conditions there.