[Auditbeat] fim(kprobes): enrich file events by coupling add_process_metadata processor

[pkoutsovasilis]

Proposed commit message

This PR adds reporting of process.group.id, process.group.name and process.entity_id in add_process_metadata processor. Also it changes the factory of MetricSets to allow the latter to specify Processors after successful instantiation; this is required as FIM has 3 different available backends, namely fsnotify, kprobes, ebpf and only the kprobes one requires to have add_process_metadata processor. Utilising the former, kprobes backend always adds a properly configured add_process_metadata processor. As a result, enriching kprobes file events with process-related data exhibits the same robustness levels of the current add_process_metadata processor. However, the current design is aligned with @nick-alayil and the sec-linux-platform which plan to increase the robustness of add_process_metadata processor in a separate effort.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• Depends on [Auditbeat] fim(ebpf): enrich file events with container id #38328

How to test this PR locally

Related issues

• Relates File Integrity Monitoring | User Information - Linux integrations#7401

Screenshots

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @pkoutsovasilis? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2024-04-13T00:10:14.568+0000

• Duration: 150 min 34 sec

Test stats 🧪

Test	Results
Failed	0
Passed	29384
Skipped	2061
Total	31445

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[andrewkroh]

That's a pretty clean approach you took to add processors to MetricSets. I think we just need a few more details in the interface godocs as mentioned in my comment.

---

I tend to view Beats as being the building block to getting data ingested how you want it. So I am not in favor of turning extra fields on by default. For example, I would rather see events starting as empty documents and the inputs contributing the raw data. Anything else is opt-in by the user through processors whether that be agent metadata, host metadata, process metadata, etc (but I'm probably not the typical user).

Today's situation is that we have a lot of things that are on by default (or not even possible to disable like host.name or agent.* or even docker metadata when run under Elastic Agent) so if you want to opt-out your only solution is to delete the fields with a processor (which is wasteful).

But I understand the desire to have everything just work out of the box. It's hard to serve the two personas that want it to just work and want to be a composable ingest tool. So allow me to mention an alternative.

1. Document within Auditbeat's FIM guide how to add the enrichment processor yourself. This gives the user full control over what fields are they wish to add. e.g.

processors:
  - if.and:
      - equals.event.module: file_integrity
      - not.has_fields: [process.name]
      - has_fields: [process.pid]
    then:
      add_process_metadata:
        match_pids:
        - process.pid

2. Add the processor to the Fleet FIM integration configuration. For a Fleet integration, the batteries should be included.

[andrewkroh]

LGTM