[META] Cluster State Publication using Remote State

[soosinha]

Please describe the end goal of this project

For remote store enabled clusters, the cluster state is being published to a remote repository. This can be used to publish the cluster state to all nodes in the cluster. Instead of sending the cluster state object over the transport layer, the cluster manager node can just send the term and version of the cluster state over the transport layer which can then be used to download the corresponding cluster state by all the nodes directly from the remote store.

Previous meta issue for 2.15 release : #13683
The current meta issue is targetted for GA release of this feature.

Supporting References

Previous meta issue for 2.15 release : #13683
RFC: #13257

Issues

• [Remote Cluster State] Send remote cluster state only when all nodes in the cluster are remote state enabled #14113
• [Remote Cluster State] Make remote publish request handler non-blocking #14102
• [Remote State] Optimize diff publication #14471
• [Remote State] Create a RemoteObjectManager interface which is used by all managers #14176
• [Remote State] Timing and tracing for RemoteWritableEntityStore #14470
• [Optimization] [Remote state] With every election full state shouldn't get uploaded to remote #10682
• [Remote Cluster State] Refactor BlobStoreformat classess #14117
• [Remote State] Use DiffUtils to find updates in Indices in RemoteClusterStateService #14177
• [BUG] [Remote store] Throttling from remote store shouldn't cause leader stepping down and re-elections #10683
• [Remote Cluster State] Ensure proper closure of IndexInput created in ChecksumBlobStoreFormat #10978
• [Remote Cluster State] Batching for reading cluster state entities from remote #14530
• [Remote State] Publish diff for index metadata #15396
• [Remote State] Add jitter to downloads from remote store #15397
• [Remote State] Send remote state location in node join flow #15406
• [Remote State] Evaluate and move remote publication to new thread pool #15437

Related component

Cluster Manager