Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add doc for debugging rest tests #14491

Merged
merged 3 commits into from
Jun 25, 2024
Merged

Conversation

bowenlan-amzn
Copy link
Member

@bowenlan-amzn bowenlan-amzn commented Jun 21, 2024

Description

Add doc for how to debug the OpenSearch cluster when running the rest tests

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • [ ] Functionality includes testing.
  • [ ] API changes companion pull request created, if applicable.
  • [ ] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

TESTING.md Outdated Show resolved Hide resolved
Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this a little clearer and add to TOC please.

TESTING.md Outdated Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
Signed-off-by: bowenlan-amzn <[email protected]>
@bowenlan-amzn
Copy link
Member Author

Let's make this a little clearer and add to TOC please.

Thanks @dblock . I update the doc, pls take another look when you have time.

@dblock dblock added the backport 2.x Backport to 2.x branch label Jun 25, 2024
@dblock dblock merged commit d320f36 into opensearch-project:main Jun 25, 2024
32 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jun 25, 2024
* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
(cherry picked from commit d320f36)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
andrross pushed a commit that referenced this pull request Jun 25, 2024
* add doc for debugging rest tests



* Update TESTING.md




* Address comment



---------



(cherry picked from commit d320f36)

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Marc Handalian <[email protected]>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this pull request Jun 27, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <[email protected]>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <[email protected]>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Added changelog

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <[email protected]>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei [email protected]

Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* Startreemapping (#24)

* Star tree mapping changes with feature flag

Signed-off-by: Bharathwaj G <[email protected]>

* Fixes and tests

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments and adding validations, integ tests

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments

Signed-off-by: Bharathwaj G <[email protected]>

---------

Signed-off-by: Bharathwaj G <[email protected]>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Bharathwaj G <[email protected]>
Signed-off-by: Sarthak Aggarwal <[email protected]>
Co-authored-by: bowenlan-amzn <[email protected]>
Co-authored-by: Prudhvi Godithi <[email protected]>
Co-authored-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: kkewwei <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Sarthak Aggarwal <[email protected]>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this pull request Jun 27, 2024
* Startreemapping (#24)

* Star tree mapping changes with feature flag

Signed-off-by: Bharathwaj G <[email protected]>

* Fixes and tests

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments and adding validations, integ tests

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments

Signed-off-by: Bharathwaj G <[email protected]>

---------

Signed-off-by: Bharathwaj G <[email protected]>

* Onheap impl startree (#25)

* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <[email protected]>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <[email protected]>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Added changelog

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <[email protected]>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei [email protected]

Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* Startreemapping (#24)

* Star tree mapping changes with feature flag

Signed-off-by: Bharathwaj G <[email protected]>

* Fixes and tests

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments and adding validations, integ tests

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments

Signed-off-by: Bharathwaj G <[email protected]>

---------

Signed-off-by: Bharathwaj G <[email protected]>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Bharathwaj G <[email protected]>
Signed-off-by: Sarthak Aggarwal <[email protected]>
Co-authored-by: bowenlan-amzn <[email protected]>
Co-authored-by: Prudhvi Godithi <[email protected]>
Co-authored-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: kkewwei <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Sarthak Aggarwal <[email protected]>

---------

Signed-off-by: Bharathwaj G <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Sarthak Aggarwal <[email protected]>
Co-authored-by: bowenlan-amzn <[email protected]>
Co-authored-by: Prudhvi Godithi <[email protected]>
Co-authored-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: kkewwei <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Sarthak Aggarwal <[email protected]>
@bowenlan-amzn bowenlan-amzn deleted the test-doc branch July 1, 2024 16:58
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this pull request Jul 10, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <[email protected]>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <[email protected]>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Added changelog

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <[email protected]>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei [email protected]

Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <[email protected]>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <[email protected]>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <[email protected]>

* changelog

Signed-off-by: panguixin <[email protected]>

* add test

Signed-off-by: panguixin <[email protected]>

---------

Signed-off-by: panguixin <[email protected]>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <[email protected]>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <[email protected]>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <[email protected]>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <[email protected]>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <[email protected]>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <[email protected]>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <[email protected]>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <[email protected]>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <[email protected]>

---------

Signed-off-by: vatsal <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <[email protected]>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <[email protected]>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <[email protected]>

* rerun security check

Signed-off-by: Peter Alfonsi <[email protected]>

---------

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <[email protected]>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Add changelog

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <[email protected]>

---------

Signed-off-by: Siddhant Deshmukh <[email protected]>
Co-authored-by: Robin Friedmann <[email protected]>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <[email protected]>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <[email protected]>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <[email protected]>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <[email protected]>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <[email protected]>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <[email protected]>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <[email protected]>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <[email protected]>

* nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <[email protected]>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <[email protected]>

* Adding tests

Signed-off-by: Bharathwaj G <[email protected]>

* Addressing comments

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments

Signed-off-by: Bharathwaj G <[email protected]>

* Star tree merge changes

Signed-off-by: Bharathwaj G <[email protected]>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <[email protected]>

* doc values file format

Signed-off-by: Sarthak Aggarwal <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: panguixin <[email protected]>
Signed-off-by: Bukhtawar Khan <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Ashish Singh <[email protected]>
Signed-off-by: Liyun Xiu <[email protected]>
Signed-off-by: vatsal <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Signed-off-by: Siddhant Deshmukh <[email protected]>
Signed-off-by: RS146BIJAY <[email protected]>
Signed-off-by: Sooraj Sinha <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sarthak Aggarwal <[email protected]>
Signed-off-by: Bharathwaj G <[email protected]>
Co-authored-by: bowenlan-amzn <[email protected]>
Co-authored-by: Prudhvi Godithi <[email protected]>
Co-authored-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: kkewwei <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
Co-authored-by: panguixin <[email protected]>
Co-authored-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <[email protected]>
Co-authored-by: Ashish Singh <[email protected]>
Co-authored-by: Liyun Xiu <[email protected]>
Co-authored-by: Vatsal <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>
Co-authored-by: Siddhant Deshmukh <[email protected]>
Co-authored-by: Robin Friedmann <[email protected]>
Co-authored-by: rishavz_sagar <[email protected]>
Co-authored-by: Sooraj Sinha <[email protected]>
Co-authored-by: Sachin Kale <[email protected]>
Co-authored-by: Sarthak Aggarwal <[email protected]>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this pull request Jul 11, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <[email protected]>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <[email protected]>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Added changelog

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <[email protected]>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei [email protected]

Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <[email protected]>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <[email protected]>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <[email protected]>

* changelog

Signed-off-by: panguixin <[email protected]>

* add test

Signed-off-by: panguixin <[email protected]>

---------

Signed-off-by: panguixin <[email protected]>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <[email protected]>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <[email protected]>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <[email protected]>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <[email protected]>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <[email protected]>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <[email protected]>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <[email protected]>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <[email protected]>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <[email protected]>

---------

Signed-off-by: vatsal <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <[email protected]>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <[email protected]>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <[email protected]>

* rerun security check

Signed-off-by: Peter Alfonsi <[email protected]>

---------

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <[email protected]>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Add changelog

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <[email protected]>

---------

Signed-off-by: Siddhant Deshmukh <[email protected]>
Co-authored-by: Robin Friedmann <[email protected]>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <[email protected]>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <[email protected]>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <[email protected]>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <[email protected]>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <[email protected]>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <[email protected]>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <[email protected]>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <[email protected]>

* nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <[email protected]>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <[email protected]>

* Adding tests

Signed-off-by: Bharathwaj G <[email protected]>

* Addressing comments

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments

Signed-off-by: Bharathwaj G <[email protected]>

* Star tree merge changes

Signed-off-by: Bharathwaj G <[email protected]>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <[email protected]>

* doc values file format

Signed-off-by: Sarthak Aggarwal <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: panguixin <[email protected]>
Signed-off-by: Bukhtawar Khan <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Ashish Singh <[email protected]>
Signed-off-by: Liyun Xiu <[email protected]>
Signed-off-by: vatsal <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Signed-off-by: Siddhant Deshmukh <[email protected]>
Signed-off-by: RS146BIJAY <[email protected]>
Signed-off-by: Sooraj Sinha <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sarthak Aggarwal <[email protected]>
Signed-off-by: Bharathwaj G <[email protected]>
Co-authored-by: bowenlan-amzn <[email protected]>
Co-authored-by: Prudhvi Godithi <[email protected]>
Co-authored-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: kkewwei <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
Co-authored-by: panguixin <[email protected]>
Co-authored-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <[email protected]>
Co-authored-by: Ashish Singh <[email protected]>
Co-authored-by: Liyun Xiu <[email protected]>
Co-authored-by: Vatsal <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>
Co-authored-by: Siddhant Deshmukh <[email protected]>
Co-authored-by: Robin Friedmann <[email protected]>
Co-authored-by: rishavz_sagar <[email protected]>
Co-authored-by: Sooraj Sinha <[email protected]>
Co-authored-by: Sachin Kale <[email protected]>
Co-authored-by: Sarthak Aggarwal <[email protected]>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this pull request Jul 11, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <[email protected]>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <[email protected]>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Added changelog

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <[email protected]>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei [email protected]

Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <[email protected]>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <[email protected]>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <[email protected]>

* changelog

Signed-off-by: panguixin <[email protected]>

* add test

Signed-off-by: panguixin <[email protected]>

---------

Signed-off-by: panguixin <[email protected]>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <[email protected]>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <[email protected]>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <[email protected]>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <[email protected]>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <[email protected]>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <[email protected]>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <[email protected]>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <[email protected]>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <[email protected]>

Signed-off-by: vatsal <[email protected]>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <[email protected]>

---------

Signed-off-by: vatsal <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <[email protected]>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <[email protected]>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <[email protected]>

* rerun security check

Signed-off-by: Peter Alfonsi <[email protected]>

---------

Signed-off-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <[email protected]>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <[email protected]>

* Add changelog

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <[email protected]>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <[email protected]>

---------

Signed-off-by: Siddhant Deshmukh <[email protected]>
Co-authored-by: Robin Friedmann <[email protected]>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <[email protected]>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <[email protected]>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <[email protected]>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <[email protected]>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <[email protected]>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <[email protected]>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <[email protected]>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <[email protected]>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <[email protected]>

* nits

Signed-off-by: Sarthak Aggarwal <[email protected]>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <[email protected]>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <[email protected]>

* Adding tests

Signed-off-by: Bharathwaj G <[email protected]>

* Addressing comments

Signed-off-by: Bharathwaj G <[email protected]>

* addressing review comments

Signed-off-by: Bharathwaj G <[email protected]>

* Star tree merge changes

Signed-off-by: Bharathwaj G <[email protected]>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <[email protected]>

* doc values file format

Signed-off-by: Sarthak Aggarwal <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: kkewwei [email protected]
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: panguixin <[email protected]>
Signed-off-by: Bukhtawar Khan <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Ashish Singh <[email protected]>
Signed-off-by: Liyun Xiu <[email protected]>
Signed-off-by: vatsal <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Signed-off-by: Siddhant Deshmukh <[email protected]>
Signed-off-by: RS146BIJAY <[email protected]>
Signed-off-by: Sooraj Sinha <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sarthak Aggarwal <[email protected]>
Signed-off-by: Bharathwaj G <[email protected]>
Co-authored-by: bowenlan-amzn <[email protected]>
Co-authored-by: Prudhvi Godithi <[email protected]>
Co-authored-by: Sagar <[email protected]>
Co-authored-by: Sagar Upadhyaya <[email protected]>
Co-authored-by: kkewwei <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
Co-authored-by: panguixin <[email protected]>
Co-authored-by: Bukhtawar Khan <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <[email protected]>
Co-authored-by: Ashish Singh <[email protected]>
Co-authored-by: Liyun Xiu <[email protected]>
Co-authored-by: Vatsal <[email protected]>
Co-authored-by: Daniel Widdis <[email protected]>
Co-authored-by: Siddhant Deshmukh <[email protected]>
Co-authored-by: Robin Friedmann <[email protected]>
Co-authored-by: rishavz_sagar <[email protected]>
Co-authored-by: Sooraj Sinha <[email protected]>
Co-authored-by: Sachin Kale <[email protected]>
Co-authored-by: Sarthak Aggarwal <[email protected]>
harshavamsi pushed a commit to harshavamsi/OpenSearch that referenced this pull request Jul 12, 2024
* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
kkewwei pushed a commit to kkewwei/OpenSearch that referenced this pull request Jul 24, 2024
…ch-project#14541)

* add doc for debugging rest tests

* Update TESTING.md

* Address comment

---------

(cherry picked from commit d320f36)

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: kkewwei <[email protected]>
wdongyu pushed a commit to wdongyu/OpenSearch that referenced this pull request Aug 22, 2024
* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <[email protected]>

* Update TESTING.md

Co-authored-by: Marc Handalian <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>

* Address comment

Signed-off-by: bowenlan-amzn <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch documentation Improvements or additions to documentation skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants