Add a disable_host option to Filebeat inputs

[andrewkroh]

What does this PR do?

This adds a configuration option publisher_pipeline.disable_host to disable the addition
of host.name in events. By default Filebeat adds host.name to all events and we want
to be able to disable this for data sources that do not originate on the host (like cloud logs).

Relates #13920

Why is it important?

ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. For some data sources Filebeat does not know that actual host and it should not use its own host name in these events. This gives a way to turn this off.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Run Filebeat and verify that the output does not have host.name.

---
filebeat.inputs:
- paths: test.log
   publisher_pipeline.disable_host: true
output.console.pretty: true

Use cases

This will be used by several modules:

• aws
• azure
• googlecloud
• misp
• o365
• okta
• panw
• suricata (depends on whether it monitors a network tap or local host traffic)
• zeek (depends on whether it monitors a network tap or local host traffic)

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request Add a disable_host option to Filebeat inputs #18159 updated]

• Start Time: 2020-05-04T13:57:33.132+0000

• Duration: 114 min 51 sec (6830831)

• Commit: 2f4c846

Test stats 🧪

Test	Results
Failed	0
Passed	7691
Skipped	1210
Total	8901

Steps errors

Expand to view the steps failures

• Name: Make -C generator/_templates/metricbeat test

  • Description: make -C generator/_templates/metricbeat test

  • Result: FAILURE

  • Duration: 2 min 27 sec<

  • Start Time: 2020-05-04T14:48:05.293+0000

• Name: Make -C generator/_templates/beat test

  • Description: make -C generator/_templates/beat test

  • Result: FAILURE

  • Duration: 1 min 51 sec<

  • Start Time: 2020-05-04T14:58:00.644+0000

• Name: Make -C generator/_templates/metricbeat test

  • Description: make -C generator/_templates/metricbeat test

  • Result: FAILURE

  • Duration: 1 min 17 sec<

  • Start Time: 2020-05-04T15:01:07.617+0000

• Name: Make -C generator/_templates/beat test

  • Description: make -C generator/_templates/beat test

  • Result: FAILURE

  • Duration: 2 min 38 sec<

  • Start Time: 2020-05-04T15:27:39.921+0000

[urso]

The change LGTM. The default filebeat configuration contains add_host_metadata. This means we will always have host set, right?

[andrewkroh]

This means we will always have host set, right?

Yes, I'm thinking to update the example config to show how put a condition on the add_host_metadata to skip based on some condition. This is the reason I opened #18148.