Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding aws-s3 metric for approximate messages waiting #34488

Merged
merged 30 commits into from
Feb 28, 2023
Merged

Adding aws-s3 metric for approximate messages waiting #34488

merged 30 commits into from
Feb 28, 2023

Conversation

kgeller
Copy link
Contributor

@kgeller kgeller commented Feb 6, 2023 •
edited
Loading

What does this PR do?

Adding the metric to answer how many message are waiting (visible) in the SQS queue

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

http://localhost:5066/dataset?pretty now includes the following

"sqs_messages_waiting_gauge": 0,

Related issues

@kgeller kgeller added enhancement Team:Security-External Integrations backport-skip Skip notification from the automated backport with mergify labels Feb 6, 2023
@kgeller kgeller self-assigned this Feb 6, 2023
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Feb 6, 2023
@mergify
Copy link
Contributor

mergify bot commented Feb 6, 2023

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b s3-messages-waiting-metric upstream/s3-messages-waiting-metric
git merge upstream/main
git push upstream s3-messages-waiting-metric

@elasticmachine
Copy link
Collaborator

elasticmachine commented Feb 6, 2023 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-02-27T21:37:49.980+0000

  • Duration: 133 min 37 sec

Test stats 🧪

Test Results
Failed 0
Passed 5740
Skipped 350
Total 6090

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@kgeller kgeller marked this pull request as ready for review February 6, 2023 21:28
@kgeller kgeller requested a review from a team as a code owner February 6, 2023 21:28
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

efd6
efd6 reviewed Feb 7, 2023
View reviewed changes
x-pack/filebeat/input/awss3/input.go Outdated
@@ -376,5 +380,16 @@ func getProviderFromDomain(endpoint string, ProviderOverride string) string {
return "unknown"
}

func PollSqsWaitingMetric(ctx context.Context, receiver *sqsReader) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be exported?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure, but I don't think so?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make it unexported.

x-pack/filebeat/input/awss3/input.go Outdated Show resolved Hide resolved
@kgeller kgeller requested a review from efd6 February 8, 2023 20:51
efd6
efd6 approved these changes Feb 8, 2023
View reviewed changes
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after nits removed.

x-pack/filebeat/input/awss3/sqs_test.go Outdated Show resolved Hide resolved
x-pack/filebeat/input/awss3/input.go Outdated
@@ -376,5 +380,16 @@ func getProviderFromDomain(endpoint string, ProviderOverride string) string {
return "unknown"
}

func PollSqsWaitingMetric(ctx context.Context, receiver *sqsReader) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make it unexported.

andrewkroh
andrewkroh reviewed Feb 9, 2023
View reviewed changes
Copy link
Member

@andrewkroh andrewkroh left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One issue I have with using the data is for the case of sqs_messages_waiting_gauge=0 , I cannot distinguish between the queue being empty and the client not having permissions to collect the value.

I think the best behavior would be not register the metric's existence until the first time it successfully obtains a value for it. @kgeller @efd6 Or can you think of a better approach?

x-pack/filebeat/docs/inputs/input-aws-s3.asciidoc Outdated Show resolved Hide resolved
x-pack/filebeat/input/awss3/metrics.go Outdated Show resolved Hide resolved
@kgeller
Copy link
Contributor Author

kgeller commented Feb 9, 2023

I think the best behavior would be not register the metric's existence until the first time it successfully obtains a value for it.

The only other approach I can think of is to register an obviously bogus count if no access, like -1. But I think that has the potential to introduce unnecessary confusion.

@efd6
Copy link
Contributor

efd6 commented Feb 9, 2023

ISTM that the zero is a zero whether it's from being unavailable or empty. I'd suggest having a second boolean metric indicating that the value is available from the data source. Is there a possibility that it would become unavailable after intially having been available? If so, then the bool should be a guage over some past window of time.

@kgeller
Copy link
Contributor Author

kgeller commented Feb 10, 2023

I pushed what Dan suggested since it helps keep the metric logic straightforward, and it keeps it super clear as to what's going on.

@kgeller
Copy link
Contributor Author

kgeller commented Feb 21, 2023

After discussing with Andrew, we tweaked the plan to be: not initializing the metric until we successfully get a value, then set the metric per the value when polled. If we lose permissions/encounter an issue, we will set the metric to -1.

@andrewkroh
Copy link
Member

andrewkroh commented Feb 21, 2023 •
edited
Loading

My main concern was usability of the data in a Lens visualization. Visualizing a single time-series will be simpler.

andrewkroh
andrewkroh requested changes Feb 22, 2023
View reviewed changes
x-pack/filebeat/input/awss3/metrics.go Outdated Show resolved Hide resolved
x-pack/filebeat/input/awss3/metrics.go Outdated Show resolved Hide resolved
x-pack/filebeat/input/awss3/metrics.go Outdated Show resolved Hide resolved
x-pack/filebeat/input/awss3/input.go Outdated Show resolved Hide resolved
x-pack/filebeat/docs/inputs/input-aws-s3.asciidoc Outdated Show resolved Hide resolved
@kgeller kgeller merged commit 2564bdc into elastic:main Feb 28, 2023
@kgeller kgeller deleted the s3-messages-waiting-metric branch February 28, 2023 14:08
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
@kgeller @chrisberkhout
Adding aws-s3 metric for approximate messages waiting (#34488)
aa5aeb7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@efd6 efd6 efd6 approved these changes

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees

@kgeller kgeller

Labels
backport-skip Skip notification from the automated backport with mergify enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@kgeller @elasticmachine @efd6 @andrewkroh