Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Allow - for source IP for AWS S3 Access pipeline #26940

Merged
merged 2 commits into from
Jul 19, 2021

Conversation

legoguy1000
Copy link
Contributor

@legoguy1000 legoguy1000 commented Jul 16, 2021

What does this PR do?

Allows for a - in place of the remote IP in the AWS S3 Access ingest pipeline.

Why is it important?

When there isn't a value for a specific field in the AWS S3 Access logs, a - is used in its place. Currently if the remote IP is not recorded and a - is present, the grok pattern doesn't match.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

How to test this PR locally

cd beats/x-pack/filebeat
TESTING_FILEBEAT_MODULES=aws TESTING_FILEBEAT_FILESETS=s3access mage -v pythonIntegTest

Related issues

Use cases

Screenshots

Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 16, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jul 16, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-07-16T21:01:37.604+0000

  • Duration: 93 min 2 sec

  • Commit: 20ae326

Test stats 🧪

Test Results
Failed 0
Passed 7872
Skipped 1201
Total 9073

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 7872
Skipped 1201
Total 9073

@kaiyan-sheng kaiyan-sheng self-assigned this Jul 16, 2021
@kaiyan-sheng kaiyan-sheng added backport-v7.14.0 Automated backport with mergify backport-v7.15.0 Automated backport with mergify bug needs_integration_sync Changes in this PR need synced to elastic/integrations. Team:Integrations Label for the Integrations team labels Jul 16, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 16, 2021
@kaiyan-sheng
Copy link
Contributor

/test

Copy link
Contributor

@kaiyan-sheng kaiyan-sheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this, looks good to me.

@legoguy1000
Copy link
Contributor Author

@kaiyan-sheng looks like it's ready to merge.

Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kaiyan-sheng kaiyan-sheng merged commit f5d33ac into elastic:master Jul 19, 2021
mergify bot pushed a commit that referenced this pull request Jul 19, 2021
mergify bot pushed a commit that referenced this pull request Jul 19, 2021
kaiyan-sheng pushed a commit that referenced this pull request Jul 19, 2021
kaiyan-sheng pushed a commit that referenced this pull request Jul 19, 2021
@kaiyan-sheng kaiyan-sheng removed the needs_integration_sync Changes in this PR need synced to elastic/integrations. label Jul 19, 2021
@legoguy1000 legoguy1000 deleted the 26913-awss3-update branch July 20, 2021 00:20
mdelapenya added a commit to mdelapenya/beats that referenced this pull request Jul 20, 2021
* master:
  Forward port 7.13.4 to master (elastic#26971)
  Use MustAddMetricSet in all metricsets (elastic#26907)
  add_process_metadata: enrich process info with process owner (elastic#21068) (elastic#21111)
  Use aws sdk paginator for FilterLogEvents and GetMetricData (elastic#26852)
  [Filebeat] Allow - for source IP for AWS S3 Access pipeline (elastic#26940)
  Increase timeout to 30secs (elastic#26841)
  Add Cluster filter on Kubernetes Overview ECS dashboard (elastic#26919)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v7.14.0 Automated backport with mergify backport-v7.15.0 Automated backport with mergify bug Team:Integrations Label for the Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Filebeat AWS Module (S3 Access Log) Grok Pattern Issue
4 participants