[Multiple Datasource] Support Amazon OpenSearch Serverless #3957

zhongnansu · 2023-04-28T21:29:05Z

Description

Support Amazon OpenSearch Serverless in SigV4 auth type for creating datasource
UI: replace "create data source" button with bottom bar
Add serverless support for legacy js client

Updated video:

iShot_2023-05-10_10.30.40.mp4

Issues Resolved

#3623 #3718 #3977

Check List

codecov-commenter · 2023-04-28T22:18:20Z

Codecov Report

Merging #3957 (eb13318) into main (c5058a3) will increase coverage by 0.01%.
The diff coverage is 55.55%.

@@            Coverage Diff             @@
##             main    #3957      +/-   ##
==========================================
+ Coverage   66.43%   66.44%   +0.01%     
==========================================
  Files        3229     3229              
  Lines       62069    62088      +19     
  Branches     9599     9605       +6     
==========================================
+ Hits        41234    41253      +19     
+ Misses      18531    18529       -2     
- Partials     2304     2306       +2

Flag	Coverage Δ
Linux	`66.38% <55.55%> (+0.01%)`	⬆️
Windows	`66.38% <55.55%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...s_credential_modal/update_aws_credential_modal.tsx	`5.00% <0.00%> (-0.27%)`	⬇️
...omponents/validation/datasource_form_validation.ts	`70.96% <0.00%> (-4.90%)`	⬇️
...rce/components/edit_form/edit_data_source_form.tsx	`66.66% <30.76%> (-1.03%)`	⬇️
...components/create_form/create_data_source_form.tsx	`74.00% <73.33%> (-1.00%)`	⬇️
...gins/data_source/server/client/configure_client.ts	`71.42% <100.00%> (ø)`
...ata_source/server/client/configure_client_utils.ts	`86.11% <100.00%> (ø)`
...ta_source/server/legacy/configure_legacy_client.ts	`71.15% <100.00%> (+11.53%)`	⬆️
...e_data_source_wizard/create_data_source_wizard.tsx	`97.56% <100.00%> (+0.06%)`	⬆️
src/plugins/data_source_management/public/types.ts	`100.00% <100.00%> (ø)`

... and 2 files with indirect coverage changes

yarn.lock

Signed-off-by: Su <[email protected]> remove expermimental text in yml Refactor create data source form for authentication Signed-off-by: Su <[email protected]>

abbyhu2000 · 2023-05-15T18:32:26Z

package.json

@@ -169,7 +169,7 @@
    "dns-sync": "^0.2.1",
    "elastic-apm-node": "^3.7.0",
    "elasticsearch": "^16.7.0",
-    "http-aws-es": "6.0.0",
+    "http-aws-es": "npm:@zhongnansu/http-aws-es@6.0.1",


why do we change the version here? are we allowed to do this?

This is a drop-in replacement as a short term solution to support serverless in legacy client, before #2220 is completed . See this comment for the reasoning and long term solution. #3623 (comment)

I've seen this kind of drop-in being done before as short term solution. Besides, the http-aws-es package is only used for data source feature

OpenSearch-Dashboards/package.json

Line 462 in 8121c9d

"vega-interpreter": "npm:@amoo-miki/[email protected]",

discussed with the team and reached an agreement to do this as a short term solution.

abbyhu2000 · 2023-05-16T21:51:32Z

Verified functionality with domain info provided by @zhongnansu

kavilla · 2023-05-17T22:55:03Z

src/plugins/data_source/server/saved_objects/data_source_saved_objects_client_wrapper.ts

@@ -320,6 +320,12 @@ export class DataSourceSavedObjectsClientWrapper {
            '"auth.credentials.region" attribute is required'
          );
        }
+
+        if (!service) {


Is multiple datasources out of experimental?

If so, then it will likely need a migration for this saved object otherwise previous saved objects will fail.

Yes, MD is out of experimental.

I don't think previous saved object will fail, because 1) we are using dynamic mapping for data source saved object. 2) service name is actually defined as optional field. Now it can still be undefined and be backwards compatible

This condition is added to the create API of saved object wrapper, to align with UI dropdown list, that a service name must be selected when creating the corresponding data source

kavilla · 2023-05-17T23:07:15Z

...management/public/components/edit_data_source/components/edit_form/edit_data_source_form.tsx

-      auth.type === AuthType.SigV4 &&
-      formValues.auth.credentials?.region !== auth.credentials?.region;
+      isAuthTypeSigV4Unchanged && formValues.auth.credentials?.region !== auth.credentials?.region;
+    const isServiceNameChanged =


i needed to change the service name once to be able to save.

No you don't. This is for the edit flow, if anything changes(title, auth type, description, etc), it will trigger the save button to render. Service name change could be one trigger, but not the mandatory

I think I know what you mean. The issue is not with the edit flow, it's with the create flow. I'll update the create flow, to have default "service name" value set to the state when "authType" changes to SigV4.

cc: @abbyhu2000

updated with new commit

Signed-off-by: Su <[email protected]>

seanneumann · 2023-05-19T16:58:38Z

@zengyan-amazon and @kristenTian can you own the reviews for this? @abbyhu2000 @kavilla they can own it.

opensearch-trigger-bot · 2023-05-19T18:51:51Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-3957-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e73779084f34b28e2cf69f28a8563e44e09f168d
# Push it to GitHub
git push --set-upstream origin backport/backport-3957-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-3957-to-2.x.

…h-project#3957) * [Multiple Datasource]Support Amazon OpenSearch Serverless in SigV4 * remove experimental text in yml * Refactor create data source form for authentication Signed-off-by: Su <[email protected]> (cherry picked from commit e737790)

…4066) * [Multiple Datasource]Support Amazon OpenSearch Serverless in SigV4 * remove experimental text in yml * Refactor create data source form for authentication Signed-off-by: Su <[email protected]> (cherry picked from commit e737790)

* Fix header icon (#3910) (#3915) * fixes header change * Update src/core/public/chrome/ui/header/header_help_menu.tsx * fixes snapshots --------- (cherry picked from commit 3cca088) Signed-off-by: Ashwin P Chandran <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Josh Romero <[email protected]> * Add server side private IP blocking for data source endpoints validation (#3912) Signed-off-by: Kristen Tian <[email protected]> * Docs (Jest): Update jest documentation links (#3931) Signed-off-by: Josh Romero <[email protected]> * Revert "[CCI] Replace jquery usage in console plugin with native methods (#3733)" (#3929) This reverts commit ffe4556. * [BUG][Dashboard listing] push to history if dashboard otherwise nav (#3922) History push will just to the current route. However, dashboardsProvider was implemented with the expectation that it was a different app. So when a plugin registered it was attempting to navigate to `app/dashboard#/app/{url}` Add tests and extra data test subject. Signed-off-by: Kawika Avilla <[email protected]> * remove jquery console release note for #3929 revert (#3930) Signed-off-by: Josh Romero <[email protected]> Co-authored-by: Ashwin P Chandran <[email protected]> * [CCI] Update js-yaml to v4.0.5 (#3770) * Update js-yaml to 4.0.5 (#3659) * Update CHANGELOG.md (#3659) Co-authored-by: Sergey Myssak <[email protected]> Signed-off-by: Andrey Myssak <[email protected]> --------- Signed-off-by: Andrey Myssak <[email protected]> Signed-off-by: Josh Romero <[email protected]> Co-authored-by: Sergey Myssak <[email protected]> Co-authored-by: Josh Romero <[email protected]> * Update README.md (#3788) * Update README.md Signed-off-by: Melissa Vagi <[email protected]> * Update README.md Co-authored-by: Miki <[email protected]> --------- Signed-off-by: Melissa Vagi <[email protected]> Co-authored-by: Miki <[email protected]> Co-authored-by: Miki <[email protected]> * Bump yaml to 2.2.2 (#3947) Signed-off-by: Manasvini B Suryanarayana <[email protected]> Co-authored-by: Sean Neumann <[email protected]> * Bump `joi` to v14 to avoid the possibility of prototype poisoning in a nested dependency (#3952) Signed-off-by: Miki <[email protected]> * [Doc] Add communication guide (#3837) * docs(COMMUNICATION): Add communication guide with info on slack, forum, and developer office hours link from README, CONTRIBUTING, DEVELOPER_GUIDE Signed-off-by: Josh Romero <[email protected]> --------- Signed-off-by: Josh Romero <[email protected]> * Temporarily hardcode chromedriver to 112.0.0 to enable all ftr tests (#3976) The latest version of chromedriver is 112.0.1 which does not support node 14. This PR hardcodes chromedriver to 112.0.0 temporarily. Pls revert it once we bump to node 18. Issue Resolved #3975 Signed-off-by: ananzh <[email protected]> * Fix wording and duplicate code in embeddable example plugin (#3911) * Fix wording and duplicate code in embeddable example plugin Signed-off-by: abbyhu2000 <[email protected]> * Fix some wording in the embeddable readme Signed-off-by: abbyhu2000 <[email protected]> --------- Signed-off-by: abbyhu2000 <[email protected]> * [CI] setup Chrome and utilize binary path (#3997) Within the CI, the virtual runner that we are utilizing has Chrome installed already. The version of Chrome is installed periodically. The most recent version of Chrome requires updates to dependencies that drop support for Node 14. This downloads chrome in the CI and then checks the chromedriver from the environment variable `TEST_BROWSER_BINARY_PATH`. Signed-off-by: Kawika Avilla <[email protected]> * [Dashboards listing] fix listing limit (#4021) Initial page size was passed to the search function instead of the listing limit causing the max amount received to be significantly less than the previously implementation. Saved objects per page is `20` by default and the listing limit per page is `1000` by default. Issue: #4017 Signed-off-by: Kawika Avilla <[email protected]> * [CCI] Fix EUI/OUI type errors (#3798) * Update find_test_subject imports for tests Signed-off-by: Alexei Karikov <[email protected]> * Update to available imports for findTestSubject Signed-off-by: Alexei Karikov <[email protected]> * Fix available import for Query and custom icon Signed-off-by: Alexei Karikov <[email protected]> * Add changelog entry Signed-off-by: Alexei Karikov <[email protected]> * Add ts-ignore Signed-off-by: Alexei Karikov <[email protected]> --------- Signed-off-by: Alexei Karikov <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> * Fix bottom bar visibility using create portal (#3336) (#3978) Signed-off-by: Sergey Myssak <[email protected]> Co-authored-by: Andrey Myssak <[email protected]> * Adds threshold to code coverage changes for project (#4040) * Fixes code coverage workflow failures for the project test due to inderect flakey changes Signed-off-by: Ashwin P Chandran <[email protected]> * Adds changelog Signed-off-by: Ashwin P Chandran <[email protected]> --------- Signed-off-by: Ashwin P Chandran <[email protected]> * Updates PR template for screenshots and test instructions (#4042) Signed-off-by: Ashwin P Chandran <[email protected]> * Replace re2 with RegExp in timeline and add unit tests (#3908) Remove re2 usage and replace it with JavaScript built-in RegExp object. Also add more unit tests to make sure that using RegExp has same expressions as using re2 library. Issue Resolve #3901 Signed-off-by: Anan Zhuang <[email protected]> * [Console] [CCI] Remove unused ul element and its custom styling. (#3993) * remove unused ul element Signed-off-by: Sirazh Gabdullin <[email protected]> * Update CHANGELOG.md Signed-off-by: Sirazh Gabdullin <[email protected]> --------- Signed-off-by: Sirazh Gabdullin <[email protected]> * Add 1.3.10 release note (#4060) (#4063) * Add release note for 1.3.10 * Address comments and add one CVE PR --------- (cherry picked from commit 4371587) Signed-off-by: abbyhu2000 <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [Multiple Datasource] Support Amazon OpenSearch Serverless (#3957) * [Multiple Datasource]Support Amazon OpenSearch Serverless in SigV4 * remove experimental text in yml * Refactor create data source form for authentication Signed-off-by: Su <[email protected]> * Remove Sass from `tile_map` plugin (#4110) * Remove Sass from tile_map plugin Signed-off-by: Matt Provost <[email protected]> * Update changelog Signed-off-by: Matt Provost <[email protected]> --------- Signed-off-by: Matt Provost <[email protected]> * Design for New Saved Object Service Interface for Custom Repository (#3954) * Adds design document for new saved object service interface for custom repository Signed-off-by: Bandini Bhopi <[email protected]> * enhance grouping for context menu options (#3924) * enhance grouping for context menu options * build panels tests and more comments Signed-off-by: David Sinclair <[email protected]> --------- Signed-off-by: David Sinclair <[email protected]> Signed-off-by: David Sinclair <[email protected]> Signed-off-by: Josh Romero <[email protected]> Co-authored-by: Josh Romero <[email protected]> * Adding Tao and Zilong to MAINTAINERS (#4137) * Adding Tao and Zilong to MAINTAINERS Signed-off-by: Yan Zeng <[email protected]> * [MD]Update data-test-subj for functional tests & fix bug in edit flow (#4126) Signed-off-by: Su <[email protected]> * Add support for Node.js >=14.20.1 <19 (#4071) * Bump Node.js requirements to 18 Signed-off-by: Miki <[email protected]> * Replace `lmdb-store` with `lmdb` Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Bump `elastic-apm-node` to the latest minor Signed-off-by: Miki <[email protected]> * Replace webpack and plugins with a patched version that uses xxhash64 * Use `xxhash64` as the hashing algorithm of webpack * Upgrade `globby` * Remove `fibers` Signed-off-by: Miki <[email protected]> * Replace `fs.rmdir` with `fs.rm` in cross-platform tests Signed-off-by: Miki <[email protected]> * Increase listener limit Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Add promise-stripping serializer Signed-off-by: Miki <[email protected]> * Bump heap for CI Signed-off-by: Miki <[email protected]> * Correct use of fs/promises in @osd/pm Signed-off-by: Miki <[email protected]> * Use fs/promise in plugin post-install cleanup Signed-off-by: Miki <[email protected]> * Set the test server's host to `0.0.0.0` Signed-off-by: Miki <[email protected]> * Sync `.node-version` file Signed-off-by: Miki <[email protected]> * Support both `isPrimary`, for Node 18, and `isMaster`, for Node 14 Signed-off-by: Miki <[email protected]> * Add types when using `isDeepStrictEqual` Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Add names to `SchemaError` to log more specific errors Signed-off-by: Miki <[email protected]> * Fix failing vega visualization tests outside the CI Signed-off-by: Miki <[email protected]> * Fix snapshot of errors thrown for undefined accessors Signed-off-by: Miki <[email protected]> * Fix flakiness of log_rotator Signed-off-by: Miki <[email protected]> * Fix asynchronous `fs` usafe in plugin discover Signed-off-by: Miki <[email protected]> * Fix mocks in @osd/optimizer Signed-off-by: Miki <[email protected]> * Fix memory leaks caused by setting states on unloaded components Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Bump Node in Dockerfile Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Remove the response `close` event as an indicator of the requesting finishing #3601 (comment) Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * [BWC] Timeout after 3 mins of waiting for OSD to be running in tests Signed-off-by: Miki <[email protected]> * Make build use the same node version that tests are run against Signed-off-by: Miki <[email protected]> * Make Node resolve DNS by IPv4 first * This is helpful to resolve `locahost` to `127.0.0.1` Signed-off-by: Miki <[email protected]> * Standardize patterns used by plugin discovery * Enhance absolute path serialization on Windows Signed-off-by: Miki <[email protected]> * Mock fetch in SenseEditor tests Signed-off-by: Miki <[email protected]> * Restore node-sass usage to fix build performance * `sass-loader@10` is the last version that supports webpack@4 * `sass` is extremely slow when using the legacy API (`render`) and to use the "Modern API" (`compileStringAsync`), `sass-loader@13` would be needed. * The performance of `sass@10` is made acceptable only with `fibers` but that is deprecated and doesn't work on Node 18 Signed-off-by: Anan Zhuang <[email protected]> Signed-off-by: Miki <[email protected]> * Revert "[CI] setup Chrome and utilize binary path (#3997)" This reverts commit 0188d05 Signed-off-by: Miki <[email protected]> * Prevent fast-fail while running functional test in CI Signed-off-by: Miki <[email protected]> * Revert "Temporarily hardcode chromedriver to 112.0.0 to enable all ftr tests (#3976)" This reverts commit 5ea0cbe. Signed-off-by: Miki <[email protected]> * Save Cypress results artifacts during CI Signed-off-by: Miki <[email protected]> * Add missing required dependency on `set-value` * Also force all to ^4.1.0 due to a vulnerability fixed in 3.1.0. Signed-off-by: Miki <[email protected]> * Prevent multiple calls to bootstrap's shutdown Signed-off-by: Miki <[email protected]> * Use Node 18.16.0 in distributions * Bump jest-canvas-mock to fix failing tests * Extend Node engines versions Signed-off-by: Miki <[email protected]> * Normalize test snapshots across Node 14, 16, and 18 Signed-off-by: Miki <[email protected]> * Update CHANGELOG for Node.js >=14.20.1 <19 support Signed-off-by: Miki <[email protected]> --------- Signed-off-by: Miki <[email protected]> Signed-off-by: Miki <[email protected]> Signed-off-by: Anan Zhuang <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Remove timeline application (#3971) * Remove timeline application In this PR, we made the following changes: First of all, clean out some advanced settings specific to timeline application and tests. * Remove timelion:default_rows: This setting defines the default number of rows that a new Timelion sheet should have. * Remove timelion:default_rows: This setting defines the default number of columns that a new Timelion sheet should have. * Remove timelion:showTutorial. Second, remove src/plugin/timeline completely and modify timeline vis. Third, remove all the functional tests related to timeline application. Issue resolve #3519 #3593 Signed-off-by: ananzh <[email protected]> --------- Signed-off-by: Anan Zhuang <[email protected]> Signed-off-by: ananzh <[email protected]> * Use `exec` in the CLI shell scripts to prevent new process creation (#3955) Signed-off-by: Miki <[email protected]> * chore (lychee): Add company.net to exclusion list (#4171) Signed-off-by: Josh Romero <[email protected]> * Bundle Node 14 as a fallback for operating systems that cannot run Node 18 (#4151) Signed-off-by: ananzh <[email protected]> Signed-off-by: Miki <[email protected]> * Refactor authentication description message (#4179) resolves #4173 Signed-off-by: Su <[email protected]> --------- Signed-off-by: Ashwin P Chandran <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Josh Romero <[email protected]> Signed-off-by: Kawika Avilla <[email protected]> Signed-off-by: Andrey Myssak <[email protected]> Signed-off-by: Melissa Vagi <[email protected]> Signed-off-by: Manasvini B Suryanarayana <[email protected]> Signed-off-by: Miki <[email protected]> Signed-off-by: ananzh <[email protected]> Signed-off-by: abbyhu2000 <[email protected]> Signed-off-by: Alexei Karikov <[email protected]> Signed-off-by: Sergey Myssak <[email protected]> Signed-off-by: Anan Zhuang <[email protected]> Signed-off-by: Sirazh Gabdullin <[email protected]> Signed-off-by: Su <[email protected]> Signed-off-by: Matt Provost <[email protected]> Signed-off-by: Bandini Bhopi <[email protected]> Signed-off-by: David Sinclair <[email protected]> Signed-off-by: David Sinclair <[email protected]> Signed-off-by: Yan Zeng <[email protected]> Signed-off-by: Miki <[email protected]> Signed-off-by: Ashish Agrawal <[email protected]> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Josh Romero <[email protected]> Co-authored-by: Kristen Tian <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Ashwin P Chandran <[email protected]> Co-authored-by: Andrey Myssak <[email protected]> Co-authored-by: Sergey Myssak <[email protected]> Co-authored-by: Melissa Vagi <[email protected]> Co-authored-by: Miki <[email protected]> Co-authored-by: Miki <[email protected]> Co-authored-by: Manasvini B Suryanarayana <[email protected]> Co-authored-by: Sean Neumann <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> Co-authored-by: Alexei Karikov <[email protected]> Co-authored-by: Andrey Myssak <[email protected]> Co-authored-by: Sirazh Gabdullin <[email protected]> Co-authored-by: Zhongnan Su <[email protected]> Co-authored-by: Matt Provost <[email protected]> Co-authored-by: Bandini <[email protected]> Co-authored-by: David Sinclair <[email protected]> Co-authored-by: Yan Zeng <[email protected]>

* Fix header icon (#3910) (#3915) * fixes header change * Update src/core/public/chrome/ui/header/header_help_menu.tsx * fixes snapshots --------- (cherry picked from commit 3cca088) Signed-off-by: Ashwin P Chandran <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Josh Romero <[email protected]> * Add server side private IP blocking for data source endpoints validation (#3912) Signed-off-by: Kristen Tian <[email protected]> * Docs (Jest): Update jest documentation links (#3931) Signed-off-by: Josh Romero <[email protected]> * Revert "[CCI] Replace jquery usage in console plugin with native methods (#3733)" (#3929) This reverts commit ffe4556. * [BUG][Dashboard listing] push to history if dashboard otherwise nav (#3922) History push will just to the current route. However, dashboardsProvider was implemented with the expectation that it was a different app. So when a plugin registered it was attempting to navigate to `app/dashboard#/app/{url}` Add tests and extra data test subject. Signed-off-by: Kawika Avilla <[email protected]> * remove jquery console release note for #3929 revert (#3930) Signed-off-by: Josh Romero <[email protected]> Co-authored-by: Ashwin P Chandran <[email protected]> * [CCI] Update js-yaml to v4.0.5 (#3770) * Update js-yaml to 4.0.5 (#3659) * Update CHANGELOG.md (#3659) Co-authored-by: Sergey Myssak <[email protected]> Signed-off-by: Andrey Myssak <[email protected]> --------- Signed-off-by: Andrey Myssak <[email protected]> Signed-off-by: Josh Romero <[email protected]> Co-authored-by: Sergey Myssak <[email protected]> Co-authored-by: Josh Romero <[email protected]> * Update README.md (#3788) * Update README.md Signed-off-by: Melissa Vagi <[email protected]> * Update README.md Co-authored-by: Miki <[email protected]> --------- Signed-off-by: Melissa Vagi <[email protected]> Co-authored-by: Miki <[email protected]> Co-authored-by: Miki <[email protected]> * Bump yaml to 2.2.2 (#3947) Signed-off-by: Manasvini B Suryanarayana <[email protected]> Co-authored-by: Sean Neumann <[email protected]> * Bump `joi` to v14 to avoid the possibility of prototype poisoning in a nested dependency (#3952) Signed-off-by: Miki <[email protected]> * [Doc] Add communication guide (#3837) * docs(COMMUNICATION): Add communication guide with info on slack, forum, and developer office hours link from README, CONTRIBUTING, DEVELOPER_GUIDE Signed-off-by: Josh Romero <[email protected]> --------- Signed-off-by: Josh Romero <[email protected]> * Temporarily hardcode chromedriver to 112.0.0 to enable all ftr tests (#3976) The latest version of chromedriver is 112.0.1 which does not support node 14. This PR hardcodes chromedriver to 112.0.0 temporarily. Pls revert it once we bump to node 18. Issue Resolved #3975 Signed-off-by: ananzh <[email protected]> * Fix wording and duplicate code in embeddable example plugin (#3911) * Fix wording and duplicate code in embeddable example plugin Signed-off-by: abbyhu2000 <[email protected]> * Fix some wording in the embeddable readme Signed-off-by: abbyhu2000 <[email protected]> --------- Signed-off-by: abbyhu2000 <[email protected]> * [CI] setup Chrome and utilize binary path (#3997) Within the CI, the virtual runner that we are utilizing has Chrome installed already. The version of Chrome is installed periodically. The most recent version of Chrome requires updates to dependencies that drop support for Node 14. This downloads chrome in the CI and then checks the chromedriver from the environment variable `TEST_BROWSER_BINARY_PATH`. Signed-off-by: Kawika Avilla <[email protected]> * [Dashboards listing] fix listing limit (#4021) Initial page size was passed to the search function instead of the listing limit causing the max amount received to be significantly less than the previously implementation. Saved objects per page is `20` by default and the listing limit per page is `1000` by default. Issue: #4017 Signed-off-by: Kawika Avilla <[email protected]> * [CCI] Fix EUI/OUI type errors (#3798) * Update find_test_subject imports for tests Signed-off-by: Alexei Karikov <[email protected]> * Update to available imports for findTestSubject Signed-off-by: Alexei Karikov <[email protected]> * Fix available import for Query and custom icon Signed-off-by: Alexei Karikov <[email protected]> * Add changelog entry Signed-off-by: Alexei Karikov <[email protected]> * Add ts-ignore Signed-off-by: Alexei Karikov <[email protected]> --------- Signed-off-by: Alexei Karikov <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> * Fix bottom bar visibility using create portal (#3336) (#3978) Signed-off-by: Sergey Myssak <[email protected]> Co-authored-by: Andrey Myssak <[email protected]> * Adds threshold to code coverage changes for project (#4040) * Fixes code coverage workflow failures for the project test due to inderect flakey changes Signed-off-by: Ashwin P Chandran <[email protected]> * Adds changelog Signed-off-by: Ashwin P Chandran <[email protected]> --------- Signed-off-by: Ashwin P Chandran <[email protected]> * Updates PR template for screenshots and test instructions (#4042) Signed-off-by: Ashwin P Chandran <[email protected]> * Replace re2 with RegExp in timeline and add unit tests (#3908) Remove re2 usage and replace it with JavaScript built-in RegExp object. Also add more unit tests to make sure that using RegExp has same expressions as using re2 library. Issue Resolve #3901 Signed-off-by: Anan Zhuang <[email protected]> * [Console] [CCI] Remove unused ul element and its custom styling. (#3993) * remove unused ul element Signed-off-by: Sirazh Gabdullin <[email protected]> * Update CHANGELOG.md Signed-off-by: Sirazh Gabdullin <[email protected]> --------- Signed-off-by: Sirazh Gabdullin <[email protected]> * Add 1.3.10 release note (#4060) (#4063) * Add release note for 1.3.10 * Address comments and add one CVE PR --------- (cherry picked from commit 4371587) Signed-off-by: abbyhu2000 <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [Multiple Datasource] Support Amazon OpenSearch Serverless (#3957) * [Multiple Datasource]Support Amazon OpenSearch Serverless in SigV4 * remove experimental text in yml * Refactor create data source form for authentication Signed-off-by: Su <[email protected]> * Remove Sass from `tile_map` plugin (#4110) * Remove Sass from tile_map plugin Signed-off-by: Matt Provost <[email protected]> * Update changelog Signed-off-by: Matt Provost <[email protected]> --------- Signed-off-by: Matt Provost <[email protected]> * Design for New Saved Object Service Interface for Custom Repository (#3954) * Adds design document for new saved object service interface for custom repository Signed-off-by: Bandini Bhopi <[email protected]> * enhance grouping for context menu options (#3924) * enhance grouping for context menu options * build panels tests and more comments Signed-off-by: David Sinclair <[email protected]> --------- Signed-off-by: David Sinclair <[email protected]> Signed-off-by: David Sinclair <[email protected]> Signed-off-by: Josh Romero <[email protected]> Co-authored-by: Josh Romero <[email protected]> * Adding Tao and Zilong to MAINTAINERS (#4137) * Adding Tao and Zilong to MAINTAINERS Signed-off-by: Yan Zeng <[email protected]> * [MD]Update data-test-subj for functional tests & fix bug in edit flow (#4126) Signed-off-by: Su <[email protected]> * Add support for Node.js >=14.20.1 <19 (#4071) * Bump Node.js requirements to 18 Signed-off-by: Miki <[email protected]> * Replace `lmdb-store` with `lmdb` Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Bump `elastic-apm-node` to the latest minor Signed-off-by: Miki <[email protected]> * Replace webpack and plugins with a patched version that uses xxhash64 * Use `xxhash64` as the hashing algorithm of webpack * Upgrade `globby` * Remove `fibers` Signed-off-by: Miki <[email protected]> * Replace `fs.rmdir` with `fs.rm` in cross-platform tests Signed-off-by: Miki <[email protected]> * Increase listener limit Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Add promise-stripping serializer Signed-off-by: Miki <[email protected]> * Bump heap for CI Signed-off-by: Miki <[email protected]> * Correct use of fs/promises in @osd/pm Signed-off-by: Miki <[email protected]> * Use fs/promise in plugin post-install cleanup Signed-off-by: Miki <[email protected]> * Set the test server's host to `0.0.0.0` Signed-off-by: Miki <[email protected]> * Sync `.node-version` file Signed-off-by: Miki <[email protected]> * Support both `isPrimary`, for Node 18, and `isMaster`, for Node 14 Signed-off-by: Miki <[email protected]> * Add types when using `isDeepStrictEqual` Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Add names to `SchemaError` to log more specific errors Signed-off-by: Miki <[email protected]> * Fix failing vega visualization tests outside the CI Signed-off-by: Miki <[email protected]> * Fix snapshot of errors thrown for undefined accessors Signed-off-by: Miki <[email protected]> * Fix flakiness of log_rotator Signed-off-by: Miki <[email protected]> * Fix asynchronous `fs` usafe in plugin discover Signed-off-by: Miki <[email protected]> * Fix mocks in @osd/optimizer Signed-off-by: Miki <[email protected]> * Fix memory leaks caused by setting states on unloaded components Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Bump Node in Dockerfile Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Remove the response `close` event as an indicator of the requesting finishing #3601 (comment) Signed-off-by: Miki <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * [BWC] Timeout after 3 mins of waiting for OSD to be running in tests Signed-off-by: Miki <[email protected]> * Make build use the same node version that tests are run against Signed-off-by: Miki <[email protected]> * Make Node resolve DNS by IPv4 first * This is helpful to resolve `locahost` to `127.0.0.1` Signed-off-by: Miki <[email protected]> * Standardize patterns used by plugin discovery * Enhance absolute path serialization on Windows Signed-off-by: Miki <[email protected]> * Mock fetch in SenseEditor tests Signed-off-by: Miki <[email protected]> * Restore node-sass usage to fix build performance * `sass-loader@10` is the last version that supports webpack@4 * `sass` is extremely slow when using the legacy API (`render`) and to use the "Modern API" (`compileStringAsync`), `sass-loader@13` would be needed. * The performance of `sass@10` is made acceptable only with `fibers` but that is deprecated and doesn't work on Node 18 Signed-off-by: Anan Zhuang <[email protected]> Signed-off-by: Miki <[email protected]> * Revert "[CI] setup Chrome and utilize binary path (#3997)" This reverts commit 0188d05 Signed-off-by: Miki <[email protected]> * Prevent fast-fail while running functional test in CI Signed-off-by: Miki <[email protected]> * Revert "Temporarily hardcode chromedriver to 112.0.0 to enable all ftr tests (#3976)" This reverts commit 5ea0cbe. Signed-off-by: Miki <[email protected]> * Save Cypress results artifacts during CI Signed-off-by: Miki <[email protected]> * Add missing required dependency on `set-value` * Also force all to ^4.1.0 due to a vulnerability fixed in 3.1.0. Signed-off-by: Miki <[email protected]> * Prevent multiple calls to bootstrap's shutdown Signed-off-by: Miki <[email protected]> * Use Node 18.16.0 in distributions * Bump jest-canvas-mock to fix failing tests * Extend Node engines versions Signed-off-by: Miki <[email protected]> * Normalize test snapshots across Node 14, 16, and 18 Signed-off-by: Miki <[email protected]> * Update CHANGELOG for Node.js >=14.20.1 <19 support Signed-off-by: Miki <[email protected]> --------- Signed-off-by: Miki <[email protected]> Signed-off-by: Miki <[email protected]> Signed-off-by: Anan Zhuang <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> * Remove timeline application (#3971) * Remove timeline application In this PR, we made the following changes: First of all, clean out some advanced settings specific to timeline application and tests. * Remove timelion:default_rows: This setting defines the default number of rows that a new Timelion sheet should have. * Remove timelion:default_rows: This setting defines the default number of columns that a new Timelion sheet should have. * Remove timelion:showTutorial. Second, remove src/plugin/timeline completely and modify timeline vis. Third, remove all the functional tests related to timeline application. Issue resolve #3519 #3593 Signed-off-by: ananzh <[email protected]> --------- Signed-off-by: Anan Zhuang <[email protected]> Signed-off-by: ananzh <[email protected]> * Use `exec` in the CLI shell scripts to prevent new process creation (#3955) Signed-off-by: Miki <[email protected]> * chore (lychee): Add company.net to exclusion list (#4171) Signed-off-by: Josh Romero <[email protected]> * Bundle Node 14 as a fallback for operating systems that cannot run Node 18 (#4151) Signed-off-by: ananzh <[email protected]> Signed-off-by: Miki <[email protected]> * Refactor authentication description message (#4179) resolves #4173 Signed-off-by: Su <[email protected]> * [CI] skip checksum verification for cypress tests (#4188) Snapshot checksum verification caused failure in test runs: #4187 Skipping the verification to enable the tests run as the snapshot of OpenSearch should not impact the tests. Issue: n/a Signed-off-by: Kawika Avilla <[email protected]> * Adds plugin manifest config to define OpenSearch plugin dependency and verifies if it is installed (#3116) Resolves Issue -#2799 Signed-off-by: Manasvini B Suryanarayana <[email protected]> * [Table Visualization] Remove custom styling for text-align:center in favor of OUI utility class. (#4164) * remove custom styling in favor of oui utility class Signed-off-by: Sirazh Gabdullin <[email protected]> * Update CHANGELOG.md Signed-off-by: Sirazh Gabdullin <[email protected]> --------- Signed-off-by: Sirazh Gabdullin <[email protected]> * Add new MAINTAINERS to CODEOWNERS file (#4199) * Add new code owners Signed-off-by: Tao Liu <[email protected]> * modify changelog.md Signed-off-by: Tao Liu <[email protected]> --------- Signed-off-by: Tao Liu <[email protected]> * Add 2.8.0 release notes (#4204) * Add 2.8.0 release notes Co-authored-by: Josh Romero <[email protected]> Signed-off-by: Kawika Avilla <[email protected]> * Chore(CHANGELOG): Update with 2.7, 2.8 releases (#3890) * Chore(CHANGELOG): Update with 2.7 release * align changelog with 2.8 release notes * update 2.8 release notes * add 1.3.10 release notes to changelog --------- Signed-off-by: Josh Romero <[email protected]> * [Saved Object Service] Adds Repository Factory Provider (#4149) * Adds Repository Factory Provider Signed-off-by: Bandini Bhopi <[email protected]> * add category option for context menus (#4144) * enhance grouping for context menu options Signed-off-by: David Sinclair <[email protected]> * change log Signed-off-by: David Sinclair <[email protected]> * remove type export Signed-off-by: David Sinclair <[email protected]> * revert border and prevent destroy options Signed-off-by: David Sinclair <[email protected]> * update comments for building panels Signed-off-by: David Sinclair <[email protected]> * build panels tests and more comments Signed-off-by: David Sinclair <[email protected]> * add category option for context menus Signed-off-by: David Sinclair <[email protected]> * changelog Signed-off-by: David Sinclair <[email protected]> * add order to groups Signed-off-by: David Sinclair <[email protected]> * documentation, shorter copyrighty, minor cleanup Signed-off-by: David Sinclair <[email protected]> * changelog Signed-off-by: David Sinclair <[email protected]> --------- Signed-off-by: David Sinclair <[email protected]> Signed-off-by: David Sinclair <[email protected]> Signed-off-by: Ashish Agrawal <[email protected]> Co-authored-by: Ashish Agrawal <[email protected]> * [CCI] Add bluebird replaces for src/plugins/saved_objects (#4026) * Add bluebird replaces for src/plugins/saved_objects * Add changelog entry --------- Signed-off-by: Alexei Karikov <[email protected]> * Validate and correct change log after 2.8 release (#4275) Signed-off-by: Su <[email protected]> --------- Signed-off-by: Ashwin P Chandran <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Josh Romero <[email protected]> Signed-off-by: Kawika Avilla <[email protected]> Signed-off-by: Andrey Myssak <[email protected]> Signed-off-by: Melissa Vagi <[email protected]> Signed-off-by: Manasvini B Suryanarayana <[email protected]> Signed-off-by: Miki <[email protected]> Signed-off-by: ananzh <[email protected]> Signed-off-by: abbyhu2000 <[email protected]> Signed-off-by: Alexei Karikov <[email protected]> Signed-off-by: Sergey Myssak <[email protected]> Signed-off-by: Anan Zhuang <[email protected]> Signed-off-by: Sirazh Gabdullin <[email protected]> Signed-off-by: Su <[email protected]> Signed-off-by: Matt Provost <[email protected]> Signed-off-by: Bandini Bhopi <[email protected]> Signed-off-by: David Sinclair <[email protected]> Signed-off-by: David Sinclair <[email protected]> Signed-off-by: Yan Zeng <[email protected]> Signed-off-by: Miki <[email protected]> Signed-off-by: Tao Liu <[email protected]> Signed-off-by: Ashish Agrawal <[email protected]> Signed-off-by: Ashish Agrawal <[email protected]> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Josh Romero <[email protected]> Co-authored-by: Kristen Tian <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Ashwin P Chandran <[email protected]> Co-authored-by: Andrey Myssak <[email protected]> Co-authored-by: Sergey Myssak <[email protected]> Co-authored-by: Melissa Vagi <[email protected]> Co-authored-by: Miki <[email protected]> Co-authored-by: Miki <[email protected]> Co-authored-by: Manasvini B Suryanarayana <[email protected]> Co-authored-by: Sean Neumann <[email protected]> Co-authored-by: Anan Zhuang <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> Co-authored-by: Alexei Karikov <[email protected]> Co-authored-by: Andrey Myssak <[email protected]> Co-authored-by: Sirazh Gabdullin <[email protected]> Co-authored-by: Zhongnan Su <[email protected]> Co-authored-by: Matt Provost <[email protected]> Co-authored-by: Bandini <[email protected]> Co-authored-by: David Sinclair <[email protected]> Co-authored-by: Yan Zeng <[email protected]> Co-authored-by: Tao Liu <[email protected]>

github-actions bot added the all-star-contributor label Apr 28, 2023

zhongnansu added the multiple datasource multiple datasource project label Apr 28, 2023

zhongnansu added the ux / ui Improvements or additions to user experience, flows, components, UI elements label May 3, 2023

kristenTian reviewed May 3, 2023

View reviewed changes

yarn.lock Show resolved Hide resolved

zhongnansu added the v2.8.0 label May 4, 2023

ashwin-pc mentioned this pull request May 4, 2023

2.8.0 Platform Upgrades for Plugins (Feature Anywhere, Point in time, Node 18,+) #3981

Closed

9 tasks

zhongnansu mentioned this pull request May 4, 2023

[DOC] [Multiple Datasource] Update documentation for AWS OpenSearch Serverlesss support in 2.8 opensearch-project/documentation-website#3983

Closed

4 tasks

zhongnansu force-pushed the md-serverless branch 2 times, most recently from 9988cce to c166aaf Compare May 9, 2023 00:05

zhongnansu added the backport 2.x label May 9, 2023

zhongnansu marked this pull request as ready for review May 9, 2023 00:07

zhongnansu requested review from ananzh, kavilla, seanneumann, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon and manasvinibs as code owners May 9, 2023 00:07

zhongnansu assigned kristenTian May 9, 2023

zhongnansu requested a review from kristenTian May 10, 2023 16:56

[Multiple Datasource]Support Amazon OpenSearch Serverless in SigV4

1d347da

Signed-off-by: Su <[email protected]> remove expermimental text in yml Refactor create data source form for authentication Signed-off-by: Su <[email protected]>

zhongnansu force-pushed the md-serverless branch from 317bb91 to 1d347da Compare May 10, 2023 20:52

kristenTian previously approved these changes May 12, 2023

View reviewed changes

zhongnansu assigned abbyhu2000 May 15, 2023

Merge branch 'main' into md-serverless

2df5ad5

abbyhu2000 reviewed May 15, 2023

View reviewed changes

zhongnansu requested a review from abbyhu2000 May 17, 2023 18:56

kavilla reviewed May 17, 2023

View reviewed changes

Update auth type and credentials in CreateDataSourceForm

8451673

Signed-off-by: Su <[email protected]>

zhongnansu dismissed kristenTian’s stale review via 8451673 May 18, 2023 00:49

zhongnansu requested a review from kavilla May 18, 2023 00:50

update

11b13ea

Signed-off-by: Su <[email protected]>

abbyhu2000 assigned kavilla and unassigned abbyhu2000 May 19, 2023

Merge branch 'main' into md-serverless

eb13318

zhongnansu requested a review from kristenTian May 19, 2023 17:16

kristenTian approved these changes May 19, 2023

View reviewed changes

zengyan-amazon approved these changes May 19, 2023

View reviewed changes

zhongnansu merged commit e737790 into opensearch-project:main May 19, 2023

zhongnansu mentioned this pull request May 19, 2023

[Backport 2.x][Multiple Datasource] Support Amazon OpenSearch Serverless (#3957) #4066

Merged

AMoo-Miki added the failed backport label May 21, 2023

joshuarrrr removed the failed backport label Aug 4, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Multiple Datasource] Support Amazon OpenSearch Serverless #3957

[Multiple Datasource] Support Amazon OpenSearch Serverless #3957

zhongnansu commented Apr 28, 2023 •

edited

Loading

codecov-commenter commented Apr 28, 2023 •

edited by codecov bot

Loading

abbyhu2000 May 15, 2023

zhongnansu May 15, 2023 •

edited

Loading

abbyhu2000 May 16, 2023

abbyhu2000 commented May 16, 2023

kavilla May 17, 2023

zhongnansu May 18, 2023 •

edited

Loading

zhongnansu May 18, 2023

kavilla May 17, 2023

zhongnansu May 17, 2023

zhongnansu May 18, 2023 •

edited

Loading

zhongnansu May 18, 2023 •

edited

Loading

seanneumann commented May 19, 2023

opensearch-trigger-bot bot commented May 19, 2023

[Multiple Datasource] Support Amazon OpenSearch Serverless #3957

[Multiple Datasource] Support Amazon OpenSearch Serverless #3957

Conversation

zhongnansu commented Apr 28, 2023 • edited Loading

Description

Updated video:

Issues Resolved

Check List

codecov-commenter commented Apr 28, 2023 • edited by codecov bot Loading

Codecov Report

abbyhu2000 May 15, 2023

Choose a reason for hiding this comment

zhongnansu May 15, 2023 • edited Loading

Choose a reason for hiding this comment

abbyhu2000 May 16, 2023

Choose a reason for hiding this comment

abbyhu2000 commented May 16, 2023

kavilla May 17, 2023

Choose a reason for hiding this comment

zhongnansu May 18, 2023 • edited Loading

Choose a reason for hiding this comment

zhongnansu May 18, 2023

Choose a reason for hiding this comment

kavilla May 17, 2023

Choose a reason for hiding this comment

zhongnansu May 17, 2023

Choose a reason for hiding this comment

zhongnansu May 18, 2023 • edited Loading

Choose a reason for hiding this comment

zhongnansu May 18, 2023 • edited Loading

Choose a reason for hiding this comment

seanneumann commented May 19, 2023

opensearch-trigger-bot bot commented May 19, 2023

zhongnansu commented Apr 28, 2023 •

edited

Loading

codecov-commenter commented Apr 28, 2023 •

edited by codecov bot

Loading

zhongnansu May 15, 2023 •

edited

Loading

zhongnansu May 18, 2023 •

edited

Loading

zhongnansu May 18, 2023 •

edited

Loading

zhongnansu May 18, 2023 •

edited

Loading