Add support for Node.js >=14.20.1 <19

[AMoo-Miki]

Description

• In this PR, we used a patched Webpack amoo-miki/webpack which uses xxhash64, a fast non-cryptographic hash algorithm algorithm. This is used in Webpack 5. Since bumping to Webpack 5 is difficult for us now due to older versions of other tools, using a patched webpack provides us the benefits of using xxhash64 hash function without having to update to Webpack 5. This strategy allows us separating updates of Node.js from updates of Webpack.
• We also shift back to use node-sass. Previously, we are using dart-sass ([CVE-2022-25758] Use dart-sass instead of node-sass #2054) due to the fact that node-sass is deprecated and has security issues. To improve dart sass performance, we brought in fibers package (add node fiber to improve performance #2319). However, fibers is no longer maintained and it doesn't support node.js 18. Without fibers min builds on both Linux and Windows take over 1h, which is a major fall-back on the performance. Meanwhile, node-sass published version 8.0.0 in March, 2023 and addressed previous CVEs. Currently, we decided to use node-sass. We might switch to others, like sass-embeddable once we bump to Webpack 5 or replacing Webpack.
• Use node 18.16.0. Also add support for Node 14, 16, and 18.

Fixes #3601

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
  • yarn test:ftr
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[peterzhuamazon]

Hi @AMoo-Miki @ananzh we probably want to only have 18.16.0 in main .nvmrc, but 16 latest versions in 2.x .nvmrc, so we will not need to make significant changes before 2.8.0 release.

We can start fully switch to 18 at some point for 2.8.x, but for 2.8.0 it is too close to make all the changes. Assuming 16 would be able to build OSD after this PR merged.

Thanks.

[joshuarrrr]

This is a substantial change which touches a lot of foundational systems, although it's something we definitely need to do. Seeing the substantial work that went into resolving issues with existing tests, what validation have we done for the ~30% of code that doesn't really have adequate test coverage? I think merging our best attempt is likely the best way to continue to validate, but I suspect there will be other regressions and issues not yet identified.

[joshuarrrr]

Jest is already prone to memory leaks - won't this exacerbate that problem?

[kavilla]

+1 I also believe listeners is configurable as well? But will need to look.

[ananzh]

We have fixed the memory leak here #3601 (comment)

Increasing the limit for listeners won't itself directly cause memory leaks; it only allows more listeners to be added without warnings. The potential memory leak comes from the accumulation of listeners that aren't properly cleaned up. If removing them when they're no longer needed properly, then increasing the limit should not exacerbate memory issues.

We have a large test suite with many async operations, it's possible that you're simply exceeding the default limit of 10 listeners without actually leaking memory. So we want to catch and fix leaking memory and also increase the listeners.

[ananzh]

We could set it to 20 process.setMaxListeners(20);
If don't set it will get the following in the integ tests

(node:3286593) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 unhandledRejection listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(Use `node --trace-warnings ...` to show where the warning was created)
(node:3286593) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 warning listeners added to [process]. Use emitter.setMaxListeners() to increase limit

[AMoo-Miki]

Keep in mind that this is only within tests and tests add numerous listeners or call functions that add listeners numerous times.

[joshuarrrr]

Also note the code coverage decrease, which is currently causing the failed check.

[kavilla]

as noted. Build repo uses this file to build OpenSearch Dashboards so we should take this into consideration if the build system needs to publish CI Runner images with Node 18 installed.

[kavilla]

@ananzh are we going to create a subsequent PR for this or fix this one?

[AMoo-Miki]

main will continue to have 18; 2.8 will use 16.

[kavilla]

+1 I also believe listeners is configurable as well? But will need to look.

[kavilla]

tbh, this workaround isn't so bad. TEST_BROWSER_BINARY_PATH is already supported in our Selenium tests. Upgrade chromedriver should respect to install a compatible chromedriver for the browser path passed by the user.

[ananzh]

created a follow-up issue:
#4147

[kavilla]

Also seeing the issue with the Advanced Settings save bar not globally applied.

Different issue.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-4071-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1c0ffeeb5b306f6a850f76c77f41be3f81ddd8c2
# Push it to GitHub
git push --set-upstream origin backport/backport-4071-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4071-to-2.x.