Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filebeat - Cloud Foundry Input not handling app.id and app.name correctly for access logs #17846

Closed
bvader opened this issue Apr 20, 2020 · 3 comments
Assignees
Labels
bug Filebeat Filebeat Team:Platforms Label for the Integrations - Platforms team

Comments

@bvader
Copy link
Contributor

bvader commented Apr 20, 2020

Summary:
Filebeat : The Cloud Foundry Input and add_cloudfoundry_metadata processor are not handling / interpreting the cloudfoundry.app.id GUID correctly for the cloudfoundry.type : access

Sample Good GUID :

cloudfoundry.app.id: 4ef05e5d-d2a6-4c58-9229-d112587b74dd
cloudfoundry.app.name: cardatabase-back-end

Sample BAD GUID :
cloudfoundry.app.id: low:3046190763798940950 high:12644936126740546741

The result of this is:

  1. The meta data processor fails and so the cloudfoundry.app.name is not found, cached and populated
  2. Because cloudfoundry.app.name is not found and cached, the api call to get the metadata is called repeatedly for each access to the app and thus filebeat is adding significant log events to the overall log stream
  3. This is potentially adding undo load to the the cloud foundry meta data API

Example Data
Discover
Index Pattern : filebeat-*
url.path : *3046190763798940950* or cloudfoundry.app.id : *3046190763798940950*'

Screen Shot 2020-04-20 at 1 15 34 PM

Note both the original HTTP Accees log and the call to the Meta Data API have the malformed GUID.

Screen Shot 2020-04-20 at 1 16 37 PM

Expected Result.
The GUID is properly handled,
The cloudfoundry.app.name is available and cached
There is a reduced load on the cloudfoundry API.

@bvader bvader added bug Filebeat Filebeat labels Apr 20, 2020
@blakerouse blakerouse added the Team:Platforms Label for the Integrations - Platforms team label Apr 20, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@exekias
Copy link
Contributor

exekias commented Apr 27, 2020

@blakerouse is this fixed by #17847?

@blakerouse
Copy link
Contributor

@exekias Yes it was, forgot to add it to the PR description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Filebeat Filebeat Team:Platforms Label for the Integrations - Platforms team
Projects
None yet
Development

No branches or pull requests

5 participants