[AWS] Support VPC endpoint for aws-s3 input SQS queue url

[kaiyan-sheng]

Proposed commit message

This PR is to add support for VPC endpoint in aws-s3 input with SQS setup. AWS customers can now access Amazon Simple Queue Service (Amazon SQS) from their Amazon Virtual Private Cloud (Amazon VPC) using VPC endpoints.
Previously we only supported SQS endpoint directly in the format like this: https://sqs.{REGION_ENDPOINT}.{ENDPOINT}/{ACCOUNT_NUMBER}/{QUEUE_NAME}.
Now with VPC, we will also support endpoint format like this: https://{VPC_ENDPOINT}.sqs.{REGION_ENDPOINT}.vpce.{ENDPOINT}/{ACCOUNT_NUMBER}/{QUEUE_NAME}

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

• Closes [aws] Support VPC endpoint for aws-s3 input SQS queue url integrations#8183

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @kaiyan-sheng? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2024-03-18T22:38:08.522+0000

• Duration: 140 min 23 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3320
Skipped	180
Total	3500

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[zmoog]

LGTM

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 969f3c9

History

• 💚 Build #2272 succeeded 6e88789
• 💚 Build #2109 succeeded ac2a021
• 💚 Build #1935 succeeded fc346cb
• 💚 Build #1894 succeeded 7372c7e
• 💚 Build #1857 succeeded 4177e8a
• 💚 Build #1812 succeeded 9a4e882

cc @kaiyan-sheng

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 969f3c9

History

• 💚 Build #2265 succeeded 6e88789
• 💚 Build #2102 succeeded ac2a021
• 💚 Build #1928 succeeded fc346cb
• 💚 Build #1887 succeeded 7372c7e
• 💚 Build #1850 succeeded 4177e8a
• 💚 Build #1805 succeeded 9a4e882

cc @kaiyan-sheng

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 969f3c9

History

• 💚 Build #1428 succeeded 6e88789
• 💚 Build #1265 succeeded ac2a021
• 💚 Build #1091 succeeded fc346cb
• 💚 Build #1050 succeeded 7372c7e
• 💚 Build #1013 succeeded 4177e8a
• 💚 Build #968 succeeded 9a4e882

cc @kaiyan-sheng

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 969f3c9

History

• 💚 Build #3483 succeeded 6e88789
• 💚 Build #3316 succeeded ac2a021
• 💚 Build #3142 succeeded fc346cb
• 💚 Build #3101 succeeded 7372c7e
• 💚 Build #3064 succeeded 4177e8a
• 💚 Build #3019 succeeded 9a4e882

cc @kaiyan-sheng

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 969f3c9

History

• 💔 Build #993 failed 6e88789

cc @kaiyan-sheng

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 969f3c9

History

• 💔 Build #1035 failed 6e88789

cc @kaiyan-sheng

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 969f3c9

History

• 💚 Build #990 succeeded 6e88789

cc @kaiyan-sheng

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 969f3c9

History

• 💔 Build #1003 failed 6e88789

cc @kaiyan-sheng

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 969f3c9

Failed CI Steps

• :windows:-family/core-windows-2016 Unit Tests
• :windows:-family/core-windows-2022 Unit Tests

History

• 💔 Build #3915 failed 6e88789
• 💔 Build #3740 failed ac2a021
• 💔 Build #3566 failed fc346cb
• 💔 Build #3524 failed 7372c7e

cc @kaiyan-sheng

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 969f3c9

Failed CI Steps

• :windows: Unit Tests / family/core-windows-2016
• :windows: Unit Tests / family/core-windows-2022

History

• 💔 Build #2562 failed 6e88789
• 💔 Build #2399 failed ac2a021
• 💔 Build #2225 failed fc346cb
• 💔 Build #2183 failed 7372c7e

cc @kaiyan-sheng