[RELEASE] Release version 2.17.0

[opensearch-trigger-bot]

This is a component issue for 2.17.0.
Coming from opensearch-project/opensearch-build#4908. Please follow the following checklist.
Please refer to the DATES in that post.

How to use this issue

This Component Release Issue

This issue captures the state of the OpenSearch release, on component/plugin level; its assignee is responsible for driving the release. Please contact them or @mention them on this issue for help.
Any release related work can be linked to this issue or added as comments to create visiblity into the release status.

Release Steps

There are several steps to the release process; these steps are completed as the whole component release and components that are behind present risk to the release. The component owner resolves the tasks in this issue and communicate with the overall release owner to make sure each component are moving along as expected.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release. The most current set of dates is on the overall release issue linked at the top of this issue.

The Overall Release Issue

Linked at the top of this issue, the overall release issue captures the state of the entire OpenSearch release including references to this issue, the release owner which is the assignee is responsible for communicating the release status broadly. Please contact them or @mention them on that issue for help.

What should I do if my plugin isn't making any changes?

If including changes in this release, increment the version on 2.x branch to 2.17.0 for Min/Core, and 2.17.0.0 for components. Otherwise, keep the version number unchanged for both.

Preparation

• Assign this issue to a release owner.
• Finalize scope and feature set and update the Public Roadmap.
• All the tasks in this issue have been reviewed by the release owner.
• Create, update, triage and label all features and issues targeted for this release with v2.17.0.
• Finalize the code and create the the release branch 2.17 from the 2.x branch.

CI/CD

• All code changes for 2.17.0 are complete.
• Ensure working and passing CI.
• Check that this repo is included in the distribution manifest.

Pre-Release

• Increment the version on the parent branch to the next development iteration.
• Gather, review and publish release notes following the rules and back port it to the release branch.git-release-notes may be used to generate release notes from your commit history.
• Confirm that all changes for 2.17.0 have been merged.
• Add this repo to the manifest for the next developer iteration.

Release Testing

• Find/fix bugs using latest tarball and docker image provided in parent release issue and update the release notes if necessary.
• Code Complete: Test within the distribution, ensuring integration, backwards compatibility, and performance tests pass.
• Sanity Testing: Sanity testing and fixing of critical issues found.
• File issues for all intermittent test failures.

Release

• Complete documentation.
• Verify all issued labeled for this release are closed or labelled for the next release.
• Verify the release date mentioned in release notes is correct and matches actual release date.

Post Release

• Prepare for an eventual security fix development iteration by incrementing the version on the release branch to the next eventual patch version.
• Add this repo to the manifest of the next eventual security patch version.
• Suggest improvements to this template.
• Conduct a retrospective, and publish its results.

[dblock]

[Catch All Triage - 1, 2, 3, 4, 5]

[gaiksaya]

Hi @dblock @andrross @reta Can we get someone assigned to this issue to take care of the release activities?
Let me know if I should tag all maintainers here asking for a volunteer.
Thanks!

[reta]

Thanks @gaiksaya , sure, I can take it !

[gaiksaya]

Hi @reta
Just a reminder about release notes that needs to be added to 2.17 branch.
Can you also please check mark all the tasks from the issue description that are complete?
Thanks!

[sandervandegeijn]

Would like to note a few things:

Core

Like I mentioned here:

#11676 (comment)

While being grateful for the PR that @finnegancarroll made to prevent the node from crashing, I think searchable snapshots in it's current state doesn't meet the quality standards that the project usually strives for. The implementation is fundamentally broken and can't be relied upon to function properly. Maybe we should disable/delete it until it has been looked after.

Security analytics
This bug: opensearch-project/security-analytics#1273

Causes an ugly exception during the boot of opensearch. It's not pretty if it's still in there with another release, at every reboot of the cluster the exceptions pop up. The issue is still untriaged in the repo.

Dashboards
These issues also break frontend functionality in dashboards plugins that are shipped with the default distribution and have been open for a while:

opensearch-project/alerting-dashboards-plugin#886
opensearch-project/alerting#1354
opensearch-project/index-management-dashboards-plugin#946
opensearch-project/alerting-dashboards-plugin#760

[sandervandegeijn]

Posted on Slack but might be relevant here:

Thanks, I write this with a lot of appreciation of the hard work that has gone into the release process. This is an open source project and you can’t look a gift horse in the mouth. This comes from involvement and appreciation, if I can help I will, but please understand I’m a product owner and not a full time developer that could fix everything myself ;)

This isn’t the first time I’ve raised these specific issues on the dashboards side. It’s not that I want my specific issues to get through, but these plugins are distributed in the default images and the primary function of the plugins isn’t working as it is supposed to. It’s about the development process and maintaining quality. ImHo these kinds of bugs should take priority over new functionality, otherwise we are introducing technical debt and sacrificing maintainability in the long term.

Perfection isn’t achievable and there always will be bugs, but if the primary function of a plugin is compromised and there is no real workaround in the GUI, that’s not how it should be for a GUI app like dashboards.

On the searchable snapshots:
The worst consequences of the searchable snapshots have been mitigated by the fix @finnegancarroll did. Many thanks for that! And don’t take this the wrong way, not crashing the node is better than crashing it and the cluster, But I’m hoping for a bit more than that. This feature was announced as stable functionality and it’s not stable at all because the architecture has a flaw.

Don’t underestimate the importance of this feature, this was the reason Elastic tried to sell us on a 30+% price hike. From the label on the issue this will be fixed after 6 months which is a long time since it’s been known for more than 8 months.

I’m able and willing to help - I understand architecture and performance characteristics. We have 50-60TB of data ready to test a new implementation with. For our new SOC program we’re eventually talking about a few PB’s of data as searchable snapshots.

[sandervandegeijn]

Sorry for spamming: #15919

Causes cluster instability because of balancing issues and not being able to allocate system indices. Manual action required to lower the number of replicas for system indices to get the cluster to green state again.

[reta]

2.17.0 was released, closing this one

[sandervandegeijn]

What is the best place to get the bugs prioritized? Some of them break functionality without a viable work around.

[reta]

What is the best place to get the bugs prioritized? Some of them break functionality without a viable work around.

@sandervandegeijn please one an issue (or issues) here: https://github.com/opensearch-project/OpenSearch/issues

[sandervandegeijn]

I already reported them :) but I.e. the broken searchable snapshots implementation is on the roadmap for over 6 months while work is being done on performance optimizations. Although I like performance as much as the next guy, I dislike broken functionality and technical debt even more 😋

[reta]

Although I like performance as much as the next guy, I dislike broken functionality and technical debt even more 😋

Totally understandable