Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #24380 to 7.x: Use alias to report container image in k8s parts #24471

Merged
merged 1 commit into from
Mar 10, 2021

Conversation

ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Mar 10, 2021

Cherry-pick of PR #24380 to 7.x branch. Original message:

What does this PR do?

This PR leverages type :alias for kubernetes.container.image so as to copy the fields under ECS container.image.name.
This affects metadata collected by kubernetes autodiscover provider, add_kubernetes_metadata processor and state_container metricset.

Why is it important?

In order to have a persistent way to report ECS container fields, without the need to duplicate fields and store both which will impact ECS.

How to test this PR locally

  1. Before all test verify that fields.yml on your testing env is properly updated with the alias change and manually setup and verify that mappings are properly loaded in ES.
  2. Check that autodiscover meta work:
filebeat.autodiscover:
      providers:
        - type: kubernetes
          node: ${NODE_NAME}
          hints.enabled: true
          hints.default_config:
            type: container
            paths:
              - /var/log/containers/*${data.kubernetes.container.id}.log

Make sure that container.image.name exist in the final documents in ES and you able to search on based on it while kubernetes.container.image is not populated in the event but is searchable because of the alias.
3. Check that add_kubernetes_metadata processor works:

filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            indexers:
            - container:
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"

Make sure that container.image.name exist in the final documents in ES and you able to search on based on it while kubernetes.container.image is not populated in the event but is searchable because of the alias.

  1. Verify with state_container metricset enabled that, container.image.name exist in the final documents in ES and you able to search based on it while kubernetes.container.image is not populated in the event but is searchable because of the alias.

Related issues

Screenshots

Screenshot 2021-03-10 at 12 30 38 PM

Screenshot 2021-03-10 at 12 30 28 PM

Screenshot 2021-03-10 at 12 41 07 PM

Screenshot 2021-03-10 at 12 40 54 PM

@ChrsMark ChrsMark requested a review from a team as a code owner March 10, 2021 12:54
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 10, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 10, 2021
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #24471 opened

  • Start Time: 2021-03-10T12:56:14.718+0000

  • Duration: 158 min 5 sec

  • Commit: 9198581

Test stats 🧪

Test Results
Failed 0
Passed 45414
Skipped 4854
Total 50268

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 45414
Skipped 4854
Total 50268

@ChrsMark ChrsMark merged commit eeb4a21 into elastic:7.x Mar 10, 2021
@zube zube bot removed the [zube]: Done label Jun 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Team:Integrations Label for the Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants