Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decommission/init merge #86

Open
wants to merge 288 commits into
base: main
Choose a base branch
from
Open

Decommission/init merge #86

wants to merge 288 commits into from

Conversation

imRishN
Copy link
Owner

@imRishN imRishN commented Nov 8, 2022

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

reta and others added 30 commits October 3, 2022 14:19
Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>
* Fixing PIT flaky tests

Signed-off-by: Bharathwaj G <[email protected]>
… Jenkins build now waits for pre-commit checks to pass before attempting (opensearch-project#4660)

* Ignore all malformed objects when ignore_malformed is true (opensearch-project#4494)

Fixes a bug to not fail the entire document when "ignore_malformed" is set to true. Allowing the valid fields 
to be indexed and ignore only the malformed fields.

Signed-off-by: Hauck <[email protected]>
Signed-off-by: Braddy Yeoh <[email protected]>

* Adding javadoc check to the precommi\t GitHub Action

Signed-off-by: Braddy Yeoh <[email protected]>

* gradle check now only runs after successful gradle precommit check

Signed-off-by: Braddy Yeoh <[email protected]>

* Added to CHANGELOG

Signed-off-by: Braddy Yeoh [email protected]
Signed-off-by: Braddy Yeoh <[email protected]>

Signed-off-by: Hauck <[email protected]>
Signed-off-by: Braddy Yeoh <[email protected]>
Signed-off-by: Braddy Yeoh [email protected]
Co-authored-by: Hauck <[email protected]>
Co-authored-by: Braddy Yeoh <[email protected]>
* First draft for Windows precommit

Signed-off-by: Sarat Vemulapalli <[email protected]>

* Adding changelog

Signed-off-by: Sarat Vemulapalli <[email protected]>

Signed-off-by: Sarat Vemulapalli <[email protected]>
* First draft for Windows precommit

Signed-off-by: Sarat Vemulapalli <[email protected]>

* Adding changelog

Signed-off-by: Sarat Vemulapalli <[email protected]>

* Adding precommit for macos

Signed-off-by: Sarat Vemulapalli <[email protected]>

* Adding Changelog

Signed-off-by: Sarat Vemulapalli <[email protected]>

Signed-off-by: Sarat Vemulapalli <[email protected]>
…t#4683)

This test fails pretty reliably if I run it on repeat. I believe the
problem is that the test assumes the function will take longer than 2ms,
which is likely not a valid assumption in all cases. Fortunately, I can
pass in a zero duration which is guaranteed to timeout even if the
system clock does not advance at all.

Also moved the assertions out of the callback into the main test method,
otherwise the assertion error messages would get buried and the test
report would just show a timeout error.

Signed-off-by: Andrew Ross <[email protected]>

Signed-off-by: Andrew Ross <[email protected]>
…elyRunningAsRoot() (opensearch-project#4656)

* Bump jna from 5.11.0 to 5.12.1

Signed-off-by: Daniel Widdis <[email protected]>

* Include Windows OS in check for definitelyRunningAsRoot()

Signed-off-by: Daniel Widdis <[email protected]>

* Combine try/catch blocks

Signed-off-by: Daniel Widdis <[email protected]>

* Someone sniped my PR number between predicting and pushing

Signed-off-by: Daniel Widdis <[email protected]>

* Update SHAs

Signed-off-by: Daniel Widdis <[email protected]>

Signed-off-by: Daniel Widdis <[email protected]>
Signed-off-by: Rishikesh1159 <[email protected]>

Signed-off-by: Rishikesh1159 <[email protected]>
* Recommission API service level changes

Signed-off-by: pranikum <[email protected]>
* Weighted round-robin scheduling policy for shard coordination traffic routing

Signed-off-by: Anshu Agarwal <[email protected]>
…ject#4688)

My previous fix introduced a new race condition by making the assertions
before waiting on the latch.

Signed-off-by: Andrew Ross <[email protected]>

Signed-off-by: Andrew Ross <[email protected]>
Fixes bwc failure caused by commit
6993ac9.

Signed-off-by: Andriy Redko <[email protected]>
…tailed_error disabled (opensearch-project#4669)

* Fixed misunderstanding message 'No OpenSearchException found' when detailed_error disabled

Signed-off-by: Xue Zhou <[email protected]>
…pensearch-project#4572)

* Update access modifier to support extensibility

Change access modifier from default to protected.
This will help to build new geo based aggregation
outside OpenSearch, by keeping GeoGrid Classes as base class.

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Updated CHANGELOG

Added PR details to CHANGELOG.md

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Rename InternalGeoGridBucket to BaseGeoGridBucket

Update class names, references and comments.

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Rename InternalGeoGrid to BaseGeoGrid

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Make GridBucket classes package-private

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Remove Internal prefix from Geo Grid classes

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Update constructor and class access modifier

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Update access modifier based on usage

Made classes package private if it is not used
outside the package.

Signed-off-by: Vijayan Balasubramanian <[email protected]>

Signed-off-by: Vijayan Balasubramanian <[email protected]>
…oject#2768)

* [Remove] LegacyESVersion.V_7_0_* and V_7_1_* constants

Removes all usages of LegacyESVersion.V_7_0_ and LegacyESVersion.V_7_1 version
constants.

Signed-off-by: Nicholas Walter Knize <[email protected]>

* Rebase from main

Signed-off-by: Nicholas Walter Knize <[email protected]>

* fix serialization issue with build flavor removal

Signed-off-by: Nicholas Walter Knize <[email protected]>

* remove stale bwc test

Signed-off-by: Nicholas Walter Knize <[email protected]>

* rebase and update

Signed-off-by: Nicholas Walter Knize <[email protected]>

* cleanup

Signed-off-by: Nicholas Walter Knize <[email protected]>

* fix failing mapper test

Signed-off-by: Nicholas Walter Knize <[email protected]>

Signed-off-by: Nicholas Walter Knize <[email protected]>
…rch-project#4066)

* Bump gson from 2.9.0 to 2.9.1 in /test/fixtures/hdfs-fixture

Bumps [gson](https://github.com/google/gson) from 2.9.0 to 2.9.1.
- [Release notes](https://github.com/google/gson/releases)
- [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md)
- [Commits](google/gson@gson-parent-2.9.0...gson-parent-2.9.1)

---
updated-dependencies:
- dependency-name: com.google.code.gson:gson
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* Always auto release the flood stage block

Removes support for using a system property to disable the automatic release of
the write block applied when a node exceeds the flood-stage watermark.

Signed-off-by: Nicholas Walter Knize <[email protected]>

* update IAE message

Signed-off-by: Nicholas Walter Knize <[email protected]>

Signed-off-by: Nicholas Walter Knize <[email protected]>
* Controlling discovery for decommissioned nodes

Signed-off-by: Rishab Nahata <[email protected]>
…oject#4702)

Removes all usages of LegacyESVersion.V_7_2_ and LegacyESVersion.V_7_3 version
constants along with related ancient APIs.

Signed-off-by: Nicholas Walter Knize <[email protected]>
…oject#4704)

Removes all usages of LegacyESVersion.V_7_4_ and LegacyESVersion.V_7_5
version constants along with ancient API logic.

Signed-off-by: Nicholas Walter Knize <[email protected]>
…re (opensearch-project#4727)

* Bump protobuf-java from 3.21.2 to 3.21.7 in /test/fixtures/hdfs-fixture

Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.2 to 3.21.7.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.21.2...v3.21.7)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…roject#4732)

Refactors base action classes implemented by external plugins through
ActionPlugin from opensearch.internal to opensearch.api to signal extensibility
to third-party developers.

Signed-off-by: Nicholas Walter Knize <[email protected]>
…pensearch-project#4726)

* Bump azure-core from 1.31.0 to 1.33.0 in /plugins/repository-azure

Bumps [azure-core](https://github.com/Azure/azure-sdk-for-java) from 1.31.0 to 1.33.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-core_1.31.0...azure-core_1.33.0)

---
updated-dependencies:
- dependency-name: com.azure:azure-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…ch-project#4076)

* Bump avro from 1.11.0 to 1.11.1 in /plugins/repository-hdfs

Bumps avro from 1.11.0 to 1.11.1.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: 0a8e37a

Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: 87bf89c

gaobinlong and others added 21 commits November 30, 2022 10:30
* Add max_shard_size parameter for Shrink API

Signed-off-by: Gao Binlong <[email protected]>

* add change log

Signed-off-by: Gao Binlong <[email protected]>

* fix yaml test failed

Signed-off-by: Gao Binlong <[email protected]>

* optimize the code

Signed-off-by: Gao Binlong <[email protected]>

* fix test failed

Signed-off-by: Gao Binlong <[email protected]>

* optimize changelog & code

Signed-off-by: Gao Binlong <[email protected]>

Signed-off-by: Gao Binlong <[email protected]>
…ject#5283)

* Update Rest status for DecommissioningFailedException

Signed-off-by: Rishab Nahata <[email protected]>

* Add tests for decommission response code

Signed-off-by: Rishab Nahata <[email protected]>

* Remove unnecessary test change

Signed-off-by: Rishab Nahata <[email protected]>

* Fix spotless check

Signed-off-by: Rishab Nahata <[email protected]>

* Fix

Signed-off-by: Rishab Nahata <[email protected]>

* Add changelog

Signed-off-by: Rishab Nahata <[email protected]>

* Update changelog

Signed-off-by: Rishab Nahata <[email protected]>

Signed-off-by: Rishab Nahata <[email protected]>
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes opensearch-project#5376

Signed-off-by: Andrew Ross <[email protected]>
…roject#5439)

Add conditional check on assertNull to fix flaky tests.

Signed-off-by: Rishikesh1159 <[email protected]>
…he Mime4j 0.8.8, Apache Poi 5.2.3, Apache PdfBox 2.0.27 (opensearch-project#5448)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>
…ect#5282)

* CheckpointState enhanced to support no-op replication

Signed-off-by: Ashish Singh <[email protected]>
Co-authored-by: Bukhtawar Khan<[email protected]>
…T/test {yaml=repository_s3/20_repository_permanent_credentials/Snapshot and Restore with repository-s3 using permanent credentials} flaky: randomizing basePath (opensearch-project#5482)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>
)

Fixes the wildcard query to not normalize the pattern when case_insensitive is
set by the user. This is achieved by creating a new normalizedWildcardQuery
method so that query_string queries (which do not support case sensitivity) can
still normalize the pattern when the default analyzer is used; maintaining
existing behavior.

Signed-off-by: Nicholas Walter Knize <[email protected]>
…ch-project#4902)" (opensearch-project#5465)

This reverts commit 8c9ca4e.

It seems that this wasn't entirely the correct way and is currently
blocking us from removing the `build.sh` from the `opensearch-build`
repository (i.e. this `build.sh` here is not yet being used).
See the discussion in opensearch-project/opensearch-build#2835 for
further details.

Signed-off-by: Ralph Ursprung <[email protected]>

Signed-off-by: Ralph Ursprung <[email protected]>
Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>
* Added jackson dependency to server

Signed-off-by: Ryan Bogan <[email protected]>

* Updated CHANGELOG

Signed-off-by: Ryan Bogan <[email protected]>

* Update build.gradle files

Signed-off-by: Ryan Bogan <[email protected]>

* Add RuntimePermission to fix errors

Signed-off-by: Ryan Bogan <[email protected]>

Signed-off-by: Ryan Bogan <[email protected]>
…ze on Windows. (opensearch-project#5511)

PR opensearch-project#5397 skipped this test in @before block but still
frequently throws a TestCouldNotBeSkippedException.  This is caused by the after block still executing and throwing  an exception
while cleaning the directory created at the path in @before.  Moving the assumption to the individual test prevents this exception by ensuring the path exists.

Signed-off-by: Marc Handalian <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>
…#5347)

* Merge first batch of feature/extensions into main

Signed-off-by: Ryan Bogan <[email protected]>

* Fixed CHANGELOG

Signed-off-by: Ryan Bogan <[email protected]>

* Fixed newline errors

Signed-off-by: Ryan Bogan <[email protected]>

* Renaming and CHANGELOG fixes

Signed-off-by: Ryan Bogan <[email protected]>

* Refactor extension loading into private method

Signed-off-by: Ryan Bogan <[email protected]>

* Removed skipValidation and added connectToExtensionNode method

Signed-off-by: Ryan Bogan <[email protected]>

* Remove unnecessary feature flag calls

Signed-off-by: Ryan Bogan <[email protected]>

* Renaming and exception handling

Signed-off-by: Ryan Bogan <[email protected]>

* Change latches to CompletableFuture

Signed-off-by: Ryan Bogan <[email protected]>

* Removed unnecessary validateSettingKey call

Signed-off-by: Ryan Bogan <[email protected]>

* Fix azure-core dependency

Signed-off-by: Ryan Bogan <[email protected]>

* Update SHAs

Signed-off-by: Ryan Bogan <[email protected]>

* Remove unintended dependency changes

Signed-off-by: Ryan Bogan <[email protected]>

* Removed dynamic settings regitration, removed info() method, and added NoopExtensionsManager

Signed-off-by: Ryan Bogan <[email protected]>

* Add javadoc

Signed-off-by: Ryan Bogan <[email protected]>

* Fixed spotless failure

Signed-off-by: Ryan Bogan <[email protected]>

* Removed NoopExtensionsManager

Signed-off-by: Ryan Bogan <[email protected]>

* Added functioning NoopExtensionsManager

Signed-off-by: Ryan Bogan <[email protected]>

* Added missing javadoc

Signed-off-by: Ryan Bogan <[email protected]>

* Remove forbiddenAPI

Signed-off-by: Ryan Bogan <[email protected]>

* Fix spotless

Signed-off-by: Ryan Bogan <[email protected]>

* Change logger.info to logger.error in handleException

Signed-off-by: Ryan Bogan <[email protected]>

* Fix ExtensionsManagerTests

Signed-off-by: Ryan Bogan <[email protected]>

* Removing unrelated change

Signed-off-by: Ryan Bogan <[email protected]>

* Update SHAs

Signed-off-by: Ryan Bogan <[email protected]>

Signed-off-by: Ryan Bogan <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: 4d46a29

@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: a159da6

Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link

Gradle Check (Jenkins) Run Completed with:

  • RESULT: null ❌
  • URL:
  • CommitID: 6ce6628

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.