[S&R] Support data streams

[jloleysens]

Summary

Update SLM management to support data streams. Rest of PR description WIP.

See #65132 (comment)

How to test

1. Start Kibana & ES (start ES with a repo path: yarn es snapshot -E path.repo=/tmp/es-snaps)
2. Create a data stream (see section below for Console commands)
3. Go to snapshot and restore UI in management section
4. Create a repository (using the path provided when starting ES)
5. Create a new SLM policy (in the Policies tab) and select a subset of indices and the data stream we created (these should be in the dropdown with badges "Data stream" if you unselect "All indices and data streams" <- new copy)
6. Run the SLM policy, creating a new snapshot
7. Go to the Snapshots tab and select the snapshot that was just created
8. The details should indicate all indices + the data stream that was snapshotted
9. Restore just the data stream by selecting the Restore action on the snapshot
10. On the step logistics view of the restore process select just the data stream and provide a capture pattern "test-data-stream" and rename to "restored-test-data-stream"
11. Restore the snapshot
12. Go to Index Management and view the data streams tab (there should be 2 data streams).
13. (Optional) Go to the indices tab in Index Management and select "Show Hidden Indices" and the backing indices should show along other hidden indices.

Create a data stream

Create an index template

PUT /_index_template/test-data-stream
{
	"index_patterns": [
		"test-data-stream*"
	],
	"template": {
		"mappings": {
			"properties": {
				"@timestamp": {
					"type": "date"
				}
			}
		}
	},
	"data_stream": {
		"timestamp_field": "@timestamp"
	}
}

Create a data stream

PUT /_data_stream/test-data-stream

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Unit or functional tests were updated or added to match the most common scenarios

Screenshots

Data streams and indices

Snapshot details view

Restoring a snapshot

[jloleysens]

@elasticmachine merge upstream

[jloleysens]

@elasticmachine merge upstream

[jloleysens]

@elasticmachine merge upstream

[jloleysens]

@elasticmachine merge upstream

[jloleysens]

@elasticmachine merge upstream

[jloleysens]

Thanks for the review @jrodewig , @gchaps and @cjcenizal .

I think I have addressed all of your points, it would be great to get another pass for the copy that I added for renaming in the restore flow: Ensure that a matching index template exists for renamed data streams. and in case I missed anything 😅

@cjcenizal I think I have addressed most of your points except for your last one:

Can we change the "Index" column to "Index or data stream"?

I think in this case only indices are available which makes this view correct. It is inconsistent with the other views, but it does not look like the data streams are returned for us to display here unfortunately - there may be some way to get this that I am not aware of. I will as ES side about this.

Otherwise, would you mind taking another look?

[jrodewig]

Left a couple of non-blocking nits. LGTM.

[jrodewig]

👍

[jloleysens]

@elasticmachine merge upstream

[cjcenizal]

Code looks great! Thanks for making those copy changes. I had a number of small suggestions but nothing blocking.

[cjcenizal]

I think we should keep this ID the same, to make it easier on the translators. We also want to uncouple the ID from the content so we don't need to change the id every time the label content changes, e.g. if we decide to change this to "Index abstractions" or even back to "Indices" in the future. This suggestion applies to other i18n IDs in this PR if there are similar changes.

[jloleysens]

Hm, I was actually thinking this would make the new translation work more discoverable for the translators. They will need to revisit these translations at any rate and I think the JSON doc is pretty much auto-generated.

I'd expect at some point for a release after 7.9 we will revisit these, hopefully with data streams and indices separated. For now I think these are also nice markers in the code for where we will need to change future copy again.

[cjcenizal]

Makes sense! Would you mind pinging the i18n team about this? You raise good points, I'm just curious what the domain owners advise.

[jloleysens]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 2e46e93

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
snapshotRestore	27	+2	25

History

• 💚 Build #58180 succeeded c2fd2da
• 💚 Build #57963 succeeded 04f60e1
• 💔 Build #57862 failed 4a93135
• 💔 Build #57831 failed edadf62
• 💔 Build #57791 failed c0a1a12

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream