-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Request] Fine grained control of (ingest|search) processors to install #14439
Labels
enhancement
Enhancement or improvement to existing feature or request
ingest-pipeline
Plugins
Search
Search query, autocomplete ...etc
v2.16.0
Issues and PRs related to version 2.16.0
v3.0.0
Issues and PRs related to version 3.0.0
Comments
andrross
added
enhancement
Enhancement or improvement to existing feature or request
untriaged
labels
Jun 18, 2024
andrross
changed the title
[Feature Request] <title>
[Feature Request] Fine grained control of (ingest|search) processors to install
Jun 18, 2024
andrross
added
Indexing
Indexing, Bulk Indexing and anything related to indexing
Plugins
labels
Jun 18, 2024
getsaurabh02
added
ingest-pipeline
and removed
Indexing
Indexing, Bulk Indexing and anything related to indexing
untriaged
labels
Jun 19, 2024
Untriaged as part of the Search Community. We discussed high level approaches on providing a static control on allowed list of processors, which can then be further expanded to dynamic with access controls on who can add/update them |
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 20, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 20, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 20, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
This was referenced Jun 20, 2024
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 20, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 20, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 25, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 26, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
that referenced
this issue
Jun 26, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to #14439 Signed-off-by: Andrew Ross <[email protected]>
opensearch-trigger-bot bot
pushed a commit
that referenced
this issue
Jun 26, 2024
Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to #14439 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit a99b494) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 26, 2024
Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 26, 2024
Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
3 tasks
reta
pushed a commit
that referenced
this issue
Jun 26, 2024
…4561) * Add allowlist setting for ingest-common processors (#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to #14439 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit a99b494) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix test issues due to class renaming on main Signed-off-by: Andrew Ross <[email protected]> --------- Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Ross <[email protected]>
akolarkunnu
pushed a commit
to akolarkunnu/OpenSearch
that referenced
this issue
Jun 27, 2024
…t#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
to andrross/OpenSearch
that referenced
this issue
Jun 27, 2024
Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
andrross
added a commit
that referenced
this issue
Jun 27, 2024
Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to #14439 Signed-off-by: Andrew Ross <[email protected]>
opensearch-trigger-bot bot
pushed a commit
that referenced
this issue
Jun 27, 2024
Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to #14439 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit d9e9944) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
reta
pushed a commit
that referenced
this issue
Jun 27, 2024
…14582) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to #14439 (cherry picked from commit d9e9944) Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
reta
added
v3.0.0
Issues and PRs related to version 3.0.0
v2.16.0
Issues and PRs related to version 2.16.0
labels
Jun 27, 2024
bharath-techie
added a commit
to bharath-techie/OpenSearch
that referenced
this issue
Jul 10, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <[email protected]> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <[email protected]> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <[email protected]> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <[email protected]> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <[email protected]> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <[email protected]> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <[email protected]> * Added changelog Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comments Signed-off-by: Sagar Upadhyaya <[email protected]> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <[email protected]> * minor UT refactor Signed-off-by: Sagar Upadhyaya <[email protected]> --------- Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei <[email protected]> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <[email protected]> * Update TESTING.md Co-authored-by: Marc Handalian <[email protected]> Signed-off-by: bowenlan-amzn <[email protected]> * Address comment Signed-off-by: bowenlan-amzn <[email protected]> --------- Signed-off-by: bowenlan-amzn <[email protected]> Co-authored-by: Marc Handalian <[email protected]> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <[email protected]> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Andrew Ross <[email protected]> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comment Signed-off-by: Sagar Upadhyaya <[email protected]> --------- Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <[email protected]> * changelog Signed-off-by: panguixin <[email protected]> * add test Signed-off-by: panguixin <[email protected]> --------- Signed-off-by: panguixin <[email protected]> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <[email protected]> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <[email protected]> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Andriy Redko <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <[email protected]> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <[email protected]> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <[email protected]> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <[email protected]> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <[email protected]> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:update changelog Signed-off-by: vatsal <[email protected]> --------- Signed-off-by: vatsal <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Co-authored-by: Daniel Widdis <[email protected]> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <[email protected]> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <[email protected]> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <[email protected]> * rerun security check Signed-off-by: Peter Alfonsi <[email protected]> --------- Signed-off-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <[email protected]> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Add changelog Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove unused logger Signed-off-by: Siddhant Deshmukh <[email protected]> * Spotless apply Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <[email protected]> --------- Signed-off-by: Siddhant Deshmukh <[email protected]> Co-authored-by: Robin Friedmann <[email protected]> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <[email protected]> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <[email protected]> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <[email protected]> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <[email protected]> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <[email protected]> * addressed nits Signed-off-by: Sarthak Aggarwal <[email protected]> * addressed major nits Signed-off-by: Sarthak Aggarwal <[email protected]> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <[email protected]> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <[email protected]> * addressing review comments Signed-off-by: Sarthak Aggarwal <[email protected]> * rebasing with main Signed-off-by: Sarthak Aggarwal <[email protected]> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <[email protected]> * nits Signed-off-by: Sarthak Aggarwal <[email protected]> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <[email protected]> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <[email protected]> * Adding tests Signed-off-by: Bharathwaj G <[email protected]> * Addressing comments Signed-off-by: Bharathwaj G <[email protected]> * addressing review comments Signed-off-by: Bharathwaj G <[email protected]> * Star tree merge changes Signed-off-by: Bharathwaj G <[email protected]> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <[email protected]> * doc values file format Signed-off-by: Sarthak Aggarwal <[email protected]> --------- Signed-off-by: bowenlan-amzn <[email protected]> Signed-off-by: Prudhvi Godithi <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar <[email protected]> Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei <[email protected]> Signed-off-by: Peter Alfonsi <[email protected]> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: panguixin <[email protected]> Signed-off-by: Bukhtawar Khan <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Shivansh Arora <[email protected]> Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Liyun Xiu <[email protected]> Signed-off-by: vatsal <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Siddhant Deshmukh <[email protected]> Signed-off-by: RS146BIJAY <[email protected]> Signed-off-by: Sooraj Sinha <[email protected]> Signed-off-by: Sachin Kale <[email protected]> Signed-off-by: Sarthak Aggarwal <[email protected]> Signed-off-by: Bharathwaj G <[email protected]> Co-authored-by: bowenlan-amzn <[email protected]> Co-authored-by: Prudhvi Godithi <[email protected]> Co-authored-by: Sagar <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> Co-authored-by: kkewwei <[email protected]> Co-authored-by: Marc Handalian <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: panguixin <[email protected]> Co-authored-by: Bukhtawar Khan <[email protected]> Co-authored-by: Andriy Redko <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <[email protected]> Co-authored-by: Ashish Singh <[email protected]> Co-authored-by: Liyun Xiu <[email protected]> Co-authored-by: Vatsal <[email protected]> Co-authored-by: Daniel Widdis <[email protected]> Co-authored-by: Siddhant Deshmukh <[email protected]> Co-authored-by: Robin Friedmann <[email protected]> Co-authored-by: rishavz_sagar <[email protected]> Co-authored-by: Sooraj Sinha <[email protected]> Co-authored-by: Sachin Kale <[email protected]> Co-authored-by: Sarthak Aggarwal <[email protected]>
bharath-techie
added a commit
to bharath-techie/OpenSearch
that referenced
this issue
Jul 11, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <[email protected]> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <[email protected]> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <[email protected]> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <[email protected]> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <[email protected]> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <[email protected]> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <[email protected]> * Added changelog Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comments Signed-off-by: Sagar Upadhyaya <[email protected]> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <[email protected]> * minor UT refactor Signed-off-by: Sagar Upadhyaya <[email protected]> --------- Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei <[email protected]> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <[email protected]> * Update TESTING.md Co-authored-by: Marc Handalian <[email protected]> Signed-off-by: bowenlan-amzn <[email protected]> * Address comment Signed-off-by: bowenlan-amzn <[email protected]> --------- Signed-off-by: bowenlan-amzn <[email protected]> Co-authored-by: Marc Handalian <[email protected]> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <[email protected]> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Andrew Ross <[email protected]> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comment Signed-off-by: Sagar Upadhyaya <[email protected]> --------- Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <[email protected]> * changelog Signed-off-by: panguixin <[email protected]> * add test Signed-off-by: panguixin <[email protected]> --------- Signed-off-by: panguixin <[email protected]> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <[email protected]> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <[email protected]> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Andriy Redko <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <[email protected]> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <[email protected]> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <[email protected]> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <[email protected]> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <[email protected]> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:update changelog Signed-off-by: vatsal <[email protected]> --------- Signed-off-by: vatsal <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Co-authored-by: Daniel Widdis <[email protected]> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <[email protected]> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <[email protected]> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <[email protected]> * rerun security check Signed-off-by: Peter Alfonsi <[email protected]> --------- Signed-off-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <[email protected]> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Add changelog Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove unused logger Signed-off-by: Siddhant Deshmukh <[email protected]> * Spotless apply Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <[email protected]> --------- Signed-off-by: Siddhant Deshmukh <[email protected]> Co-authored-by: Robin Friedmann <[email protected]> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <[email protected]> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <[email protected]> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <[email protected]> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <[email protected]> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <[email protected]> * addressed nits Signed-off-by: Sarthak Aggarwal <[email protected]> * addressed major nits Signed-off-by: Sarthak Aggarwal <[email protected]> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <[email protected]> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <[email protected]> * addressing review comments Signed-off-by: Sarthak Aggarwal <[email protected]> * rebasing with main Signed-off-by: Sarthak Aggarwal <[email protected]> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <[email protected]> * nits Signed-off-by: Sarthak Aggarwal <[email protected]> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <[email protected]> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <[email protected]> * Adding tests Signed-off-by: Bharathwaj G <[email protected]> * Addressing comments Signed-off-by: Bharathwaj G <[email protected]> * addressing review comments Signed-off-by: Bharathwaj G <[email protected]> * Star tree merge changes Signed-off-by: Bharathwaj G <[email protected]> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <[email protected]> * doc values file format Signed-off-by: Sarthak Aggarwal <[email protected]> --------- Signed-off-by: bowenlan-amzn <[email protected]> Signed-off-by: Prudhvi Godithi <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar <[email protected]> Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei <[email protected]> Signed-off-by: Peter Alfonsi <[email protected]> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: panguixin <[email protected]> Signed-off-by: Bukhtawar Khan <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Shivansh Arora <[email protected]> Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Liyun Xiu <[email protected]> Signed-off-by: vatsal <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Siddhant Deshmukh <[email protected]> Signed-off-by: RS146BIJAY <[email protected]> Signed-off-by: Sooraj Sinha <[email protected]> Signed-off-by: Sachin Kale <[email protected]> Signed-off-by: Sarthak Aggarwal <[email protected]> Signed-off-by: Bharathwaj G <[email protected]> Co-authored-by: bowenlan-amzn <[email protected]> Co-authored-by: Prudhvi Godithi <[email protected]> Co-authored-by: Sagar <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> Co-authored-by: kkewwei <[email protected]> Co-authored-by: Marc Handalian <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: panguixin <[email protected]> Co-authored-by: Bukhtawar Khan <[email protected]> Co-authored-by: Andriy Redko <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <[email protected]> Co-authored-by: Ashish Singh <[email protected]> Co-authored-by: Liyun Xiu <[email protected]> Co-authored-by: Vatsal <[email protected]> Co-authored-by: Daniel Widdis <[email protected]> Co-authored-by: Siddhant Deshmukh <[email protected]> Co-authored-by: Robin Friedmann <[email protected]> Co-authored-by: rishavz_sagar <[email protected]> Co-authored-by: Sooraj Sinha <[email protected]> Co-authored-by: Sachin Kale <[email protected]> Co-authored-by: Sarthak Aggarwal <[email protected]>
bharath-techie
added a commit
to bharath-techie/OpenSearch
that referenced
this issue
Jul 11, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <[email protected]> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <[email protected]> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <[email protected]> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <[email protected]> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <[email protected]> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <[email protected]> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <[email protected]> * Added changelog Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comments Signed-off-by: Sagar Upadhyaya <[email protected]> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <[email protected]> * minor UT refactor Signed-off-by: Sagar Upadhyaya <[email protected]> --------- Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei <[email protected]> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <[email protected]> * Update TESTING.md Co-authored-by: Marc Handalian <[email protected]> Signed-off-by: bowenlan-amzn <[email protected]> * Address comment Signed-off-by: bowenlan-amzn <[email protected]> --------- Signed-off-by: bowenlan-amzn <[email protected]> Co-authored-by: Marc Handalian <[email protected]> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <[email protected]> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Andrew Ross <[email protected]> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <[email protected]> * Addressing comment Signed-off-by: Sagar Upadhyaya <[email protected]> --------- Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <[email protected]> * changelog Signed-off-by: panguixin <[email protected]> * add test Signed-off-by: panguixin <[email protected]> --------- Signed-off-by: panguixin <[email protected]> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <[email protected]> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <[email protected]> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Andriy Redko <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <[email protected]> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <[email protected]> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <[email protected]> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <[email protected]> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <[email protected]> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <[email protected]> Signed-off-by: vatsal <[email protected]> * Fix issue 14519:update changelog Signed-off-by: vatsal <[email protected]> --------- Signed-off-by: vatsal <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Co-authored-by: Daniel Widdis <[email protected]> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <[email protected]> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <[email protected]> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <[email protected]> * rerun security check Signed-off-by: Peter Alfonsi <[email protected]> --------- Signed-off-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <[email protected]> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <[email protected]> * Add changelog Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove unused logger Signed-off-by: Siddhant Deshmukh <[email protected]> * Spotless apply Signed-off-by: Siddhant Deshmukh <[email protected]> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <[email protected]> --------- Signed-off-by: Siddhant Deshmukh <[email protected]> Co-authored-by: Robin Friedmann <[email protected]> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <[email protected]> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <[email protected]> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <[email protected]> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <[email protected]> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <[email protected]> * addressed nits Signed-off-by: Sarthak Aggarwal <[email protected]> * addressed major nits Signed-off-by: Sarthak Aggarwal <[email protected]> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <[email protected]> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <[email protected]> * addressing review comments Signed-off-by: Sarthak Aggarwal <[email protected]> * rebasing with main Signed-off-by: Sarthak Aggarwal <[email protected]> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <[email protected]> * nits Signed-off-by: Sarthak Aggarwal <[email protected]> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <[email protected]> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <[email protected]> * Adding tests Signed-off-by: Bharathwaj G <[email protected]> * Addressing comments Signed-off-by: Bharathwaj G <[email protected]> * addressing review comments Signed-off-by: Bharathwaj G <[email protected]> * Star tree merge changes Signed-off-by: Bharathwaj G <[email protected]> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <[email protected]> * doc values file format Signed-off-by: Sarthak Aggarwal <[email protected]> --------- Signed-off-by: bowenlan-amzn <[email protected]> Signed-off-by: Prudhvi Godithi <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: Sagar <[email protected]> Signed-off-by: kkewwei [email protected] Signed-off-by: kkewwei <[email protected]> Signed-off-by: Peter Alfonsi <[email protected]> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Sagar Upadhyaya <[email protected]> Signed-off-by: panguixin <[email protected]> Signed-off-by: Bukhtawar Khan <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Shivansh Arora <[email protected]> Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Liyun Xiu <[email protected]> Signed-off-by: vatsal <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Siddhant Deshmukh <[email protected]> Signed-off-by: RS146BIJAY <[email protected]> Signed-off-by: Sooraj Sinha <[email protected]> Signed-off-by: Sachin Kale <[email protected]> Signed-off-by: Sarthak Aggarwal <[email protected]> Signed-off-by: Bharathwaj G <[email protected]> Co-authored-by: bowenlan-amzn <[email protected]> Co-authored-by: Prudhvi Godithi <[email protected]> Co-authored-by: Sagar <[email protected]> Co-authored-by: Sagar Upadhyaya <[email protected]> Co-authored-by: kkewwei <[email protected]> Co-authored-by: Marc Handalian <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> Co-authored-by: Peter Alfonsi <[email protected]> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: panguixin <[email protected]> Co-authored-by: Bukhtawar Khan <[email protected]> Co-authored-by: Andriy Redko <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <[email protected]> Co-authored-by: Ashish Singh <[email protected]> Co-authored-by: Liyun Xiu <[email protected]> Co-authored-by: Vatsal <[email protected]> Co-authored-by: Daniel Widdis <[email protected]> Co-authored-by: Siddhant Deshmukh <[email protected]> Co-authored-by: Robin Friedmann <[email protected]> Co-authored-by: rishavz_sagar <[email protected]> Co-authored-by: Sooraj Sinha <[email protected]> Co-authored-by: Sachin Kale <[email protected]> Co-authored-by: Sarthak Aggarwal <[email protected]>
harshavamsi
pushed a commit
to harshavamsi/OpenSearch
that referenced
this issue
Jul 12, 2024
…t#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
harshavamsi
pushed a commit
to harshavamsi/OpenSearch
that referenced
this issue
Jul 12, 2024
…ch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
kkewwei
pushed a commit
to kkewwei/OpenSearch
that referenced
this issue
Jul 24, 2024
…ensearch-project#14561) * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]> (cherry picked from commit a99b494) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix test issues due to class renaming on main Signed-off-by: Andrew Ross <[email protected]> --------- Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Ross <[email protected]> Signed-off-by: kkewwei <[email protected]>
kkewwei
pushed a commit
to kkewwei/OpenSearch
that referenced
this issue
Jul 24, 2024
…ch-project#14562) (opensearch-project#14582) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 (cherry picked from commit d9e9944) Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: kkewwei <[email protected]>
wdongyu
pushed a commit
to wdongyu/OpenSearch
that referenced
this issue
Aug 22, 2024
…t#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
wdongyu
pushed a commit
to wdongyu/OpenSearch
that referenced
this issue
Aug 22, 2024
…ch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
enhancement
Enhancement or improvement to existing feature or request
ingest-pipeline
Plugins
Search
Search query, autocomplete ...etc
v2.16.0
Issues and PRs related to version 2.16.0
v3.0.0
Issues and PRs related to version 3.0.0
Is your feature request related to a problem? Please describe
I have a deployment of OpenSearch that uses dedicated coordinator nodes. I need to be able to strictly control the work that happens on these nodes, and in particular I want to limit CPU-bound work. Thus far I have disabled the use of search and ingest pipelines by restricting traffic to the corresponding endpoints that create pipelines. I now have the requirement to enable some pipelines (specifically the pipelines defined by the neural-search plugin). However, all default ingest and search pipelines are defined in their corresponding modules (ingest-common and search-pipeline-common), which are installed by default with no way to opt out. Because these pipelines would introduce a new class of CPU-bound workload on the coordinator nodes, I do not want these pipelines to be enabled in my installation. OpenSearch provides no way to disable the registration of these pipelines.
Describe the solution you'd like
I would like a mechanism to not install all the default search and ingest processors. This could be implemented by a new setting (one each for search and ingest). It might work like the following:
"_all"
): All default processors are installed. This is the behavior today.[]
: No default processors installed.["foo", "bar"]
: Only processors named "foo" and "bar" are installed.Related component
Search Pipelines and Ingest Pipelines
Describe alternatives you've considered
Given that all the default processors are installed via modules, and my current requirement is to install none of them, then this could also be solved by a mechanism to opt-out of certain modules all together. I think this can be done today with an
rm -rf modules/ingest-common
command after installation, but this feels a bit like a hack. Also, opting out of the modules all together is less flexible than the setting mechanism proposed above.The text was updated successfully, but these errors were encountered: