Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] journald: Make include_matches backwards compatible with 7.x config #30032

Merged

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Jan 26, 2022

What does this PR do?

To make transitioning from 7.x/8.0 to 8.1 easier allow the old format to continue working
by translating it internally to the new format. This logs a deprecation warning when the
old config format is detected.

Why is it important?

This will make it easier for Agent users to migrate from 7.x/8.0 to 8.1. There will be time window
when integrations with the old format are running with the new Agent.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Logs

{
  "@timestamp": "2022-01-26T17:26:47.443Z",
  "ecs.version": "1.6.0",
  "log.level": "warn",
  "log.logger": "cfgwarn",
  "log.origin": {
    "file.line": 97,
    "file.name": "journald/config.go"
  },
  "message": "DEPRECATED: Please migrate your journald input's include_matches config to the new more expressive format.",
  "service.name": "filebeat"
}

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 26, 2022
@mergify
Copy link
Contributor

mergify bot commented Jan 26, 2022

This pull request does not have a backport label. Could you fix it @andrewkroh? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Jan 26, 2022
@andrewkroh andrewkroh added backport-v8.0.0 Automated backport with mergify Filebeat Filebeat Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team and removed backport-skip Skip notification from the automated backport with mergify labels Jan 26, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 26, 2022
@andrewkroh andrewkroh requested a review from kvch January 26, 2022 17:34
@andrewkroh andrewkroh added backport-v8.1.0 Automated backport with mergify and removed backport-v8.0.0 Automated backport with mergify labels Jan 26, 2022
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jan 26, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-01-27T01:02:03.442+0000

  • Duration: 22 min 58 sec

  • Commit: 7031342

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

To make transitioning from 7.x/8.0 to 8.1 easier allow the old format to continue working
by translating it internally to the new format. This logs a deprecation warning when the
old config format is detected.

This will make it easier to Agent users to migrate from 7.x/8.0 to 8.1. There will be time window
when integrations with the old format are running with the new Agent (or vice versa).
@andrewkroh andrewkroh force-pushed the feature/fb/journald-include-matches-bwc branch from 819b363 to 7031342 Compare January 26, 2022 18:10
@andrewkroh
Copy link
Member Author

/test

@andrewkroh andrewkroh merged commit 70ce544 into elastic:master Jan 28, 2022
v1v added a commit to v1v/beats that referenced this pull request Jan 31, 2022
…k-version-after-8-0-creation

* upstream/master: (69 commits)
  Update stale config following (elastic#30082)
  Make include_matches backwards compatible with 7.x config (elastic#30032)
  [Filebeat] Update handling of elasticsearch server logs (elastic#30018)
  Remove SSL3 support from libbeat and its documentation. (elastic#30071)
  Revert "Packaging: rename arm64 suffix to aarch64 in the tar.gz artifacts ONLY (elastic#28813)" (elastic#30083)
  [libbeat] Add script processor to all beats (elastic#29752)
  Add fonts to support more different types of characters for multiple languages (elastic#29861)
  libbeat/reader: Fix messge conversion to beat.Event (elastic#30057)
  probot[stale]: ignore issues with the tag flaky-test (elastic#30065)
  [DOCS] Add redirect for GSuite module (elastic#30034)
  [Automation] Update elastic stack version to 8.1.0-aa69d697 for testing (elastic#30012)
  Remove msitools install for windows build, using the latest docker image with msitools preinstalled (elastic#30040)
  filebeat/generator/fields: fix dropped error (elastic#29943)
  Include the error message with auditd module events (elastic#30009)
  [Metricbeat] gcp: add firestore metricset (elastic#29918)
  probot: update stale dates (elastic#29997)
  Metricbeat enterprise search module: add xpack.enabled support (elastic#29871)
  x-pack/packetbeat: install Npcap at start-up when required (elastic#29112)
  [Filebeat] Fix panic in decode_cef when recovering from invalid data (elastic#30038)
  Correctly fixe how selected packages are defined (elastic#30039)
  ...
WildDogOne pushed a commit to WildDogOne/integrations that referenced this pull request Aug 21, 2022
Update kibana.constraint to allow 8.1.0+. 8.1.0 is required because elastic/beats#30032
is needed due to a breaking change that was introduced in 8.0.0.

Rename message_id to ECS event.code. This field was added in 8.0. From man systemd.journal-fields

  MESSAGE_ID=
           A 128-bit message identifier ID for recognizing certain
           message types, if this is desirable. ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v8.1.0 Automated backport with mergify Filebeat Filebeat Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants